auto-deep-researcher-24x7
๐ฅ An autonomous AI agent that runs your deep learning experiments 24/7 while you sleep. Zero-cost monitoring, Leader-Worker architecture, constant-size memory.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Badge for your README
Markdown
[](/listings/xiangyue-zhang--auto-deep-researcher-24x7)HTML
<a href="/listings/xiangyue-zhang--auto-deep-researcher-24x7"><img src="/api/listings/xiangyue-zhang--auto-deep-researcher-24x7/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
> **Complete beginner?** Follow every step below. You'll go from zero to a running experiment agent in ~10 minutes. > > **Prefer AI-guided setup?** Open [`AI_GUIDE.md`](AI_GUIDE.md) in Claude Code, ChatGPT, or Codex โ the AI will walk you through everything interactively.
Step 0: What You Need
| Requirement | Why | How to Check | |-------------|-----|-------------| | Python 3.10+ | Runtime | `python3 --version` | | [Claude Code](https://claude.ai/claude-code) | The AI backbone | `claude --version` | | 1+ NVIDIA GPU | For training | `nvidia-smi` | | Anthropic API key | LLM calls | `echo $ANTHROPIC_API_KEY` |
Don't have an API key? Get one at [console.anthropic.com](https://console.anthropic.com/) and set it:
export ANTHROPIC_API_KEY="sk-ant-xxxxx" # Add to ~/.bashrc or ~/.zshrc to make it permanent
Step 1: Install
# Clone the repo git clone https://github.com/Xiangyue-Zhang/auto-deep-researcher-24x7.git cd auto-deep-researcher-24x7 # Install Python dependencies pip install -r requirements.txt # Install 8 Claude slash commands and 8 Codex local skills python install.py # Verify everything works python -m core.loop --check
You should see:
Deep Researcher Agent โ Installer
========================================
โ Claude /auto-experiment
โ Claude /experiment-status
โ Claude /gpu-monitor
โ Claude /daily-papers
โ Claude /paper-analyze
โ Claude /conf-search
โ Claude /progress-report
โ Claude /obsidian-sync
โ Codex $auto-experiment
...
Done! 8 Claude commands and 8 Codex skills installed.Step 2: Create Your First Project
Let's say you want to train a ResNet on CIFAR-100. Create a project folder with a `PROJECT_BRIEF.md`:
mkdir ~/my-first-experiment cd ~/my-first-experiment
Now write the brief โ **this is the most important file**. It tells the agent what you want:
cat > PROJECT_BRIEF.md << 'EOF' # Goal Train a ResNet-50 on CIFAR-100 to reach 80%+ test accuracy. # Codebase The agent should create the training code from scratch using PyTorch. - Use torchvision for the dataset (auto-download) - Save checkpoints to ./checkpoints/ - Log metrics to ./logs/ # What to Try - Start with a basic ResNet-50, lr=0.1, SGD, 100 epochs - If accuracy < 75%, try cosine annealing + warmup - If accuracy 75-80%, try adding mixup or cutout augmentation - If accuracy > 80%, the goal is reached # Constraints - Use GPU 0 only - Max 100 epochs per run - Batch size 128 # Current Status No experiments run yet. Starting from scratch. EOF
**Tips for writing a good brief:** - Be specific about the goal (metric + target value) - Tell it where the code/data is (or say "create from scratch") - List constraints (which GPU, max epochs, etc.) - Give it a decision tree ("if X, try Y") โ this guides the agent like you would guide a junior student
Step 3: Launch the Agent
**Option A: Through Claude Code (recommended)**
Open Claude Code and type:
/auto-experiment --project ~/my-first-experiment --gpu 0
**Option B: Through Python directly**
python -m core.loop \ --project ~/my-first-experiment \ --gpu 0 \ --max-cycles 5 # Stop after 5 cycles (remove for unlimited)
Step 4: Watch What Happens
The agent will now do everything automatically. Here's what each cycle looks like:
=== Cycle 1 ===
[THINK] Reading PROJECT_BRIEF.md...
Goal: ResNet-50 on CIFAR-100, target 80%+
No previous experiments. Starting with baseline.
Plan: Basic ResNet-50, lr=0.1, SGD with momentum, 100 epochs.
[EXECUTE] Creating train.py...
Creating config.yaml...
Running dry-run (2 steps)... โ No errors
Launching training: nohup python train.py --config config.yaml
PID: 12345, Log: logs/exp001.log
[MONITOR] Training in progress... (zero LLM cost)
15:00 โ PID alive, GPU 98%, Epoch 12/100, loss=2.34What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
README.md ยท README ยท 1,204 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
- <p align="center">
- <img src="assets/banner.png" alt="Deep Researcher Agent" width="700"/>
- </p>
- <h1 align="center">Deep Researcher Agent</h1>
- <h3 align="center">24/7 Autonomous Deep Learning Experiment Agent</h3>
- <p align="center">
- <strong>An AI agent that autonomously runs your deep learning experiments 24/7 while you sleep.</strong>
- </p>
- <p align="center">
- <a href="README.md">English</a> |
- <a href="docs/README_CN.md">ไธญๆ</a> |
- <a href="docs/README_JP.md">ๆฅๆฌ่ช</a> |
- <a href="docs/README_KR.md">ํ๊ตญ์ด</a>
- </p>
- <p align="center">
- <a href="#quickstart"><img src="https://img.shields.io/badge/-Quick_Start-blue?style=for-the-badge" alt="Quick Start"/></a>
- <a href="docs/architecture.md"><img src="https://img.shields.io/badge/-Architecture-orange?style=for-the-badge" alt="Architecture"/></a>
- </p>
- <p align="center">
- <img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="Python"/>
- <img src="https://img.shields.io/badge/Claude_Code-compatible-blueviolet.svg" alt="Claude Code"/>
- <img src="https://img.shields.io/badge/Codex_CLI-compatible-green.svg" alt="Codex CLI"/>
- <img src="https://img.shields.io/badge/license-Apache_2.0-green.svg" alt="License"/>
- <a href="https://github.com/Xiangyue-Zhang/auto-deep-researcher-24x7/stargazers"><img src="https://img.shields.io/github/stars/Xiangyue-Zhang/auto-deep-researcher-24x7?color=yellow&logo=github&label=Stars" alt="Stars"/></a>
- </p>
- <p align="center">
- <a href="https://arxiv.org/abs/2604.05854"><img src="https://img.shields.io/badge/Technical%20Report-2604.05854-b31b1b.svg" alt="Technical Report"/></a>
- </p>
- ---
- ## Recent Updates
- **2026-06-03 โ Domestic LLM API presets**
- - Run the agent on a **Chinese LLM API instead of a Claude/Codex subscription** by
- setting `agent.provider` to a one-word preset โ `deepseek`, `qwen` (`dashscope`),
- `kimi` (`moonshot`), or `glm` (`zhipu`). The preset auto-fills the OpenAI-compatible
- `base_url` and the default key env (`DEEPSEEK_API_KEY` / `DASHSCOPE_API_KEY` /
- `MOONSHOT_API_KEY` / `ZHIPUAI_API_KEY`); you just set `model` to that vendor's model
- id. `base_url` / `api_key_env` stay overridable for self-hosted or proxied endpoints.
- This is a thin alias over the existing OpenAI-compatible path โ no new dependency.
- (`core/agents.py`)
- ```yaml
- agent:
- provider: "deepseek" # or qwen / kimi / glm
- model: "deepseek-chat" # vendor's model id
- ```
- **2026-06-02 โ Slurm execution backend + truthful experiment outcomes**
- - **Slurm execution backend** โ added `execution.mode: "slurm"` so the agent can
- drive experiments on a Slurm cluster. The controller stays local; training is
- submitted to the login node with `sbatch --parsable` over a single transient
- SSH call that exits immediately โ **no process is ever left running on the
- login node**. `sacct` is the sole liveness authority (Slurm enforces `--time`),
- GPU status is read from the partition's `squeue` occupancy, and two bounds
- inside the liveness check (consecutive-unknown grace + a `--time`-derived
- wall-clock backstop) guarantee the monitor loop terminates even if the cluster
- goes unreachable โ without ever reaping a job `sacct` still reports as queued
- or running. File and repo-reading ops reuse the SSH path (the login node shares
- the NFS workspace). (`core/execution.py`)
- - **Truthful experiment outcomes** โ the monitor now asks the backend for a
- finished job's real terminal state via `final_status()`, so a `FAILED` /
- `TIMEOUT` / `CANCELLED` run is no longer silently recorded as `completed`. The
- outcome flows into `state.json`, the experiment ledger, and the REFLECT
- context, so the agent reasons over what actually happened. On Slurm the state
- comes from `sacct`; pid-only backends (`local`/`ssh`) report it as
- indeterminate and keep prior behavior. (`core/monitor.py`, `core/loop.py`)
- - Additive and opt-in; `local`/`ssh` behavior is unchanged. (+21 unit tests, no
- cluster required.)
- **2026-06-01 โ v2.0 (major update)**
- This release gives the agent (a) a persistent, queryable memory of its own
- experiments, (b) explicit progress/quality/safety signals derived from that
- memory, and (c) much stronger code- and literature-reading tools. Every change
- is additive and backward-compatible โ existing projects keep working unchanged,
- the new gate and rate limit are opt-in, and the whole suite is unit-tested
- without a GPU or network (60 โ 99 tests).
- *New: autonomy layer*
- - **Experiment ledger** โ every cycle's hypothesis, metrics, and outcome are appended to `workspace/experiments.jsonl`. Crash-safe, zero token cost, and fed back into planning so the agent remembers what it already tried. (`core/ledger.py`)
- - **Data-driven stagnation signal** โ the planner is told, from the ledger's metric trajectory, whether results are still improving or have stalled (set `ledger.metric_key`), instead of only a binary repeat-counter.
- - **Append-only research journals** โ `DEAD_ENDS.md` (failed approaches โ do not retry) and `INSIGHTS.md` (durable observations). Never compacted; rotated to dated backups when large, so history is never silently dropped. (`core/journal.py`)
- - **Zero-cost violation scanner + advisory phase gate** โ surface stuck/stale states and whether a baseline metric bar is met, as pure functions over state + ledger. (`core/safety.py`, `core/ledger.py`)
- - **Proactive anti-burn rate limiting** โ optional `agent.max_cycles_per_hour` cap protects budget when the agent is stuck in a loop.
- *New: agent tools*
- - **Code comprehension** โ `search_code` (regex grep across the workspace), `list_tree` (recursive, depth-limited repo map), and `read_file` line ranges so large files are no longer blindly truncated. Symlink-safe (never escapes the workspace).
- - **Literature** โ `get_paper` (paper details + reference/citation snowballing) and `search_arxiv` (freshest preprints), alongside the existing Semantic Scholar search.
- - All new tools work identically in **local and SSH** execution modes.
- *Config:* new optional sections `ledger:`, `stagnation:`, `journal:`, `safety:`, `gates:`, and `agent.max_cycles_per_hour` โ all default to current behavior. See `config.yaml`.
- **2026-04-22**
- - Added explicit compatible-API configuration, dual Claude/Codex skill installation, and safer skill-installer ownership checks.
- **2026-04-21**
- - Added an optional SSH execution backend so the controller can stay local while code edits, training, logs, PID checks, and GPU queries run on one remote host.
- **2026-04-19**
- - Added a real multi-turn worker tool-use loop with authoritative tool-result handoff, stricter CLI behavior, and safer tool-call parsing.
- **2026-04-18**
- - Added subscription-backed `claude_cli` and `codex_cli` provider modes with fail-fast provider validation and more defensive CLI subprocess handling.
- **2026-04-09**
- - Reduced token growth and tightened loop/tool safeguards with leader-history resets, no-progress fallback, and stronger path and shell protections.
- **2026-04-08**
- - Added progress tracking exports with optional Obsidian sync and local text fallback when no vault is configured.
- ## Start In 3 Steps
- If you only want the shortest path to a working experiment loop, do this:
- 1. Create a project folder with one file: `PROJECT_BRIEF.md`
- 2. Run `/auto-experiment --project /path/to/project --gpu 0`
- 3. Check progress with `/experiment-status` or optional Obsidian/local text notes
- Prefer AI-guided setup? Open [`AI_GUIDE.md`](AI_GUIDE.md) in Claude / ChatGPT / Codex and let the assistant walk you through it.
- ## What You Actually Need
- | Requirement | Required | Notes |
- |-------------|----------|-------|
- | Python 3.10+ | Yes | Runtime |
- | 1+ NVIDIA GPU | Yes | For training |
- | API key | Yes | Anthropic-compatible or OpenAI-compatible endpoint |
- | `PROJECT_BRIEF.md` | Yes | Main control file |
- | Project `config.yaml` | Optional | Only if you want to override defaults |
- | Obsidian vault | Optional | If absent, notes fall back to local text files |
- ## Minimum Working Example
- The smallest project you can launch looks like this:
- ```text
- my-first-experiment/
- โโโ PROJECT_BRIEF.md
- โโโ workspace/ # auto-created
- ```
- Minimal `PROJECT_BRIEF.md`:
- ```md
- # Goal
- Train a ResNet-50 on CIFAR-100 to reach 80%+ accuracy.
- # Codebase
- Create the training code from scratch in PyTorch.
- # What to Try
- - Start with a basic ResNet-50 baseline.
- - If accuracy < 75%, improve optimization and schedule.
- - If accuracy is 75-80%, try augmentation.
- - If accuracy > 80%, stop and report.
- # Constraints
- - Use GPU 0 only
- - Max 100 epochs per run
- ```
- That is enough to start. Everything else is optional refinement.
- ## What This Project Is Good At
- This project is for people who already know what experiment they want to run, but do not want to babysit the loop:
- - edit code
- - launch training
- - monitor runs
- - parse logs
- - decide the next variation
- - keep going while you sleep
- It is not trying to replace the researcher. It is trying to take over the repetitive experiment-ops layer.
- ## Why It Feels Different From A Simple Script
- - It does not just launch one run. It keeps iterating.
- - It does not just monitor. It reflects and decides the next step.
- - It stays cheap because training-time monitoring makes zero LLM calls.
- - It stays controllable because the human can override direction at any cycle.
- - It now supports persistent progress notes in Obsidian or local text files.
- ## How You Stay In Control
- You control the research direction through three files:
- - `PROJECT_BRIEF.md`: stable goal, constraints, allowed search space
- - `HUMAN_DIRECTIVE.md`: temporary redirect for the next cycle
- - `workspace/MEMORY_LOG.md`: rolling memory of results and decisions
- Common control patterns:
- ```md
- # Keep the search narrow
- - Only tune augmentation.
- - Do not change the backbone.
- - Keep training budget fixed.
- ```
- ```md
- # Make the agent stop exploring a weak direction
- - If gain stays below 0.3 points for 3 runs, stop this branch.
- - Return to the last trusted baseline and try a different idea.
- ```
- ```md
- # Force result verification
- - If a result looks unusually strong, rerun with the same seed and one new seed.
- - Do not claim improvement until both reproduce.
- ```
- ## How You See Progress
- You should never have to guess what the agent is doing.
- - `/experiment-status` shows current goal, best result, cycle count, running status, and recent decisions
- - `/progress-report` generates a structured summary
- - `/obsidian-sync` refreshes persistent notes manually
- - `workspace/progress_tracking/` stores local text notes when no Obsidian vault is configured
- If you want a dashboard outside the terminal:
- ```yaml
- obsidian:
- enabled: true
- vault_path: "~/Documents/MyObsidianVault" # Optional
- auto_append_daily: true
- ```
- If `vault_path` is empty, the same information is saved locally:
- ```text
- workspace/progress_tracking/Dashboard.txt
- workspace/progress_tracking/Daily/YYYY-MM-DD.txt
- ```
- ---
- ## ๐ A Note on Why We Built This โ and How We Hope You'll Use It
- > **Our hope is simple: science stays pure, and the human stays in the loop.**
- We built this framework for one reason โ to take the *repetitive, mechanical* parts of running deep learning experiments off the researcher's plate (launching jobs, watching GPUs, parsing logs, sweeping hyperparameters) so that more of your time can go into **the part that actually matters: thinking**.
- If you're here because you want to spend less time babysitting training runs and more time reading, reasoning, and chasing your own ideas โ welcome. That's exactly who we built this for.
- **A gentle thought we'd love every user to share with us:**
- The agent is happy to run the experiments. But please let the *ideas*, the *interpretation*, and the *scientific judgment* remain yours. We don't see automation and academic integrity as being in tension โ quite the opposite. The hours this tool gives back are meant to be reinvested in **deeper thinking**, not in skipping it.
- So we'd kindly ask that this project not be used to fabricate results, to generate "research" with no human in the loop, or to shortcut the parts of science that depend on a human actually understanding what they're doing. That isn't the future we want to help build โ and we don't think it's the one most of you want either.
- > **Science should stay pure. The agent can run the experiments โ but the ideas, the interpretation, and the responsibility belong to the human.**
- >
- > **ๅญฆๆฏๅบๅฝไฟๆ็บฏ็ฒนใ** Agent ๅฏไปฅๆฟไฝ ่ทๅฎ้ช๏ผไฝ ideaใๅคๆญไธ่ดฃไปป๏ผ่ฏท็็ปไบบๆฅๆฟๆ ใๆไปฌ็ๅฟๅธๆๆฏไธไฝไฝฟ็จ่ ้ฝ่ฝ **human in the loop ๅฐๅปๆ่**๏ผๆ่ฟไธชๅทฅๅ ท็ไธๆฅ็ๆถ้ด๏ผๆๅ ฅๅฐ็ๆญฃๅฑไบไฝ ่ชๅทฑ็็ ็ฉถๆนๅ้ใ
- >
- > **็งๅญฆใฏ็ด็ฒใงใใในใใงใใ** Agent ใฏๅฎ้จใ่ตฐใใใใใจใใงใใพใใใใขใคใใขใป่งฃ้ใป่ฒฌไปปใฏใใฉใใไบบ้ใฎๆใซๆฎใใฆใใ ใใใ
- >
- > **๊ณผํ์ ์์ํด์ผ ํฉ๋๋ค.** Agent๋ ์คํ์ ๋์ ์คํํด ์ค ์ ์์ง๋ง, ์์ด๋์ด์ ํด์, ๊ทธ๋ฆฌ๊ณ ์ฑ ์์ ๋ถ๋ ์ฌ๋์ ๋ชซ์ผ๋ก ๋จ๊ฒจ์ฃผ์ธ์.
- We trust the people who pick up this tool to take that seriously โ and we built it because we believe most of you already do. Thank you for being one of them. ๐
- ---
- ## The Core Idea
- You design the experiment. The agent handles the repetitive loop.
- **Deep Researcher Agent**:
- 1. **Thinks** โ Reads your project brief, analyzes previous results, plans the next experiment
- 2. **Executes** โ Modifies code/configs, runs a dry-run, launches training on GPU
- 3. **Monitors** โ Watches training at **zero LLM cost** (just process checks + log reads)
- 4. **Reflects** โ Parses results, compares with baselines, decides what to try next
- 5. **Repeats** โ 24/7, without human intervention
- ```
- You sleep 8 hours โ Agent runs 3 experiment cycles
- You go on vacation โ Agent explores 50+ hyperparameter configs
- You write your paper โ Agent already has the results table ready
- ```
- ---
- ## Battle-Tested Results
- > Not benchmarks. Real results from months of 24/7 autonomous operation across research projects.
- | Metric | Result |
- |--------|--------|
- | Autonomous experiment cycles completed | 500+ |
- | Best single-project improvement | 52% over baseline (across 200+ auto-run experiments) |
- | Concurrent projects managed | 4 projects across 4 GPU servers |
- | Longest continuous autonomous operation | 30+ days without human intervention |
- | Average LLM cost per 24h cycle | ~$0.08 |
- ---
- ## Key Innovation: Zero-Cost Monitoring
- The #1 concern with running LLM agents 24/7: **cost**.
- Most agent frameworks call the LLM every few minutes to "check progress". That's $50+/day.
- Experiment Agent **sleeps** during training โ zero API calls. It only wakes the LLM when training finishes.
- ```
- LLM Active Zero Cost LLM Active
- โโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโ
- โ THINK โ โ TRAIN & MONITOR โ โ REFLECT โ
- โ (5-10 min) โ โ (hours/days) โ โ (5-10 min) โ
- โ โ โ โ โ โ
- โ โข Analyze โ โ โข kill -0 $PID โ โ โข Parse โ
- โ โข Plan โ โ โข nvidia-smi โ โ logs โ
- โ โข Code โ โ โข tail log โ โ โข Compare โ
- โ โ โ โ โ โข Decide โ
- โ ~$0.05 โ โ $0.00 โ โ ~$0.03 โ
- โโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโ
- ```
- **24-hour cycle with 8 hours of training: ~$0.08 in LLM calls.**
- ---
- ## Architecture
- ### The THINK โ EXECUTE โ REFLECT Loop
- ```
- โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
- โ โโโโโโโโโโโโ โโโโโโโโโโโโ โโโโโโโโโโโโ โ
- โ โ THINK โโโโโโ EXECUTE โโโโโโ REFLECT โโโโ โ
- โ โ โ โ โ โ โ โ โ
- โ โ Analyze โ โ Dry-run โ โ Evaluate โ โ โ
- โ โ Plan โ โ Launch โ โ Compare โ โ โ
- โ โ Decide โ โ Monitor โ โ Update โ โ โ
- โ โโโโโโโโโโโโ โโโโโโโโโโโโ โโโโโโโโโโโโ โ โ
- โ โ โ โ
- โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
- โ โป 24/7 Loop โ
- โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
- ```
- ### Leader-Worker Agent System
- Only ONE worker runs at a time. Others idle at zero cost.
- ```
- โโโโโโโโโโโโโโโโโ
- โ Leader โ Persistent conversation
- โ (Planner) โ within each cycle
- โโโโโฌโโโโฌโโโโฌโโโโ
- โ โ โ
- โโโโโโโโโ โ โโโโโโโโโ
- โ โ โ
- โโโโโโโโโโโโ โโโโโโโโโโโโ โโโโโโโโโโโโ
- โ Idea โ โ Code โ โ Writing โ
- โ Agent โ โ Agent โ โ Agent โ
- โ (4 tools)โ โ (5 tools)โ โ (3 tools)โ
- โโโโโโโโโโโโ โโโโโโโโโโโโ โโโโโโโโโโโโ
- ```
- ### Two-Tier Memory (Constant Size Forever)
- ```
- โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
- โ Tier 1: PROJECT_BRIEF.md โ
- โ โข Frozen project reference โ
- โ โข Max 3,000 chars โ
- โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
- โ Tier 2: MEMORY_LOG.md โ
- โ โข Key Results (auto-compact at 1,200ch) โ
- โ โข Recent Decisions (rolling last 15) โ
- โ โข Max 2,000 chars โ
- โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
- โ Total: ~5K chars / ~1,500 tokens โ
- โ SAME whether running 1 day or 6 months โ
- โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
- ```
- ### Cost Control Strategies (8 Total)
- | # | Strategy | Savings |
- |---|----------|---------|
- | 1 | Zero-LLM monitoring during training | 90%+ of runtime is free |
- | 2 | Two-Tier memory with auto-compaction | Fixed context window |
- | 3 | Leader conversation persists within cycle | Brief sent once per cycle |
- | 4 | Anthropic prompt caching | System/tools cached |
- | 5 | Per-agent minimal tool sets (3-5 tools) | Less schema overhead |
- | 6 | Slim system prompts | Fewer input tokens |
- | 7 | State trimmed before sending | No bloat |
- | 8 | Single worker at a time | No parallel LLM costs |
- ---
- <a name="quickstart"></a>
- ## Getting Started (Step by Step)
- > **Complete beginner?** Follow every step below. You'll go from zero to a running experiment agent in ~10 minutes.
- >
- > **Prefer AI-guided setup?** Open [`AI_GUIDE.md`](AI_GUIDE.md) in Claude Code, ChatGPT, or Codex โ the AI will walk you through everything interactively.
- ### Step 0: What You Need
- | Requirement | Why | How to Check |
- |-------------|-----|-------------|
- | Python 3.10+ | Runtime | `python3 --version` |
- | [Claude Code](https://claude.ai/claude-code) | The AI backbone | `claude --version` |
- | 1+ NVIDIA GPU | For training | `nvidia-smi` |
- | Anthropic API key | LLM calls | `echo $ANTHROPIC_API_KEY` |
- Don't have an API key? Get one at [console.anthropic.com](https://console.anthropic.com/) and set it:
- ```bash
- export ANTHROPIC_API_KEY="sk-ant-xxxxx"
- # Add to ~/.bashrc or ~/.zshrc to make it permanent
- ```
- ### Step 1: Install
- ```bash
- # Clone the repo
- git clone https://github.com/Xiangyue-Zhang/auto-deep-researcher-24x7.git
- cd auto-deep-researcher-24x7
- # Install Python dependencies
- pip install -r requirements.txt
- # Install 8 Claude slash commands and 8 Codex local skills
- python install.py
- # Verify everything works
- python -m core.loop --check
- ```
- You should see:
- ```
- Deep Researcher Agent โ Installer
- ========================================
- โ Claude /auto-experiment
- โ Claude /experiment-status
- โ Claude /gpu-monitor
- โ Claude /daily-papers
- โ Claude /paper-analyze
- โ Claude /conf-search
- โ Claude /progress-report
- โ Claude /obsidian-sync
- โ Codex $auto-experiment
- ...
- Done! 8 Claude commands and 8 Codex skills installed.
- ```
- ### Step 2: Create Your First Project
- Let's say you want to train a ResNet on CIFAR-100. Create a project folder with a `PROJECT_BRIEF.md`:
- ```bash
- mkdir ~/my-first-experiment
- cd ~/my-first-experiment
- ```
- Now write the brief โ **this is the most important file**. It tells the agent what you want:
- ```bash
- cat > PROJECT_BRIEF.md << 'EOF'
- # Goal
- Train a ResNet-50 on CIFAR-100 to reach 80%+ test accuracy.
- # Codebase
- The agent should create the training code from scratch using PyTorch.
- - Use torchvision for the dataset (auto-download)
- - Save checkpoints to ./checkpoints/
- - Log metrics to ./logs/
- # What to Try
- - Start with a basic ResNet-50, lr=0.1, SGD, 100 epochs
- - If accuracy < 75%, try cosine annealing + warmup
- - If accuracy 75-80%, try adding mixup or cutout augmentation
- - If accuracy > 80%, the goal is reached
- # Constraints
- - Use GPU 0 only
- - Max 100 epochs per run
- - Batch size 128
- # Current Status
- No experiments run yet. Starting from scratch.
- EOF
- ```
- **Tips for writing a good brief:**
- - Be specific about the goal (metric + target value)
- - Tell it where the code/data is (or say "create from scratch")
- - List constraints (which GPU, max epochs, etc.)
- - Give it a decision tree ("if X, try Y") โ this guides the agent like you would guide a junior student
- ### Step 3: Launch the Agent
- **Option A: Through Claude Code (recommended)**
- Open Claude Code and type:
- ```
- /auto-experiment --project ~/my-first-experiment --gpu 0
- ```
- **Option B: Through Python directly**
- ```bash
- python -m core.loop \
- --project ~/my-first-experiment \
- --gpu 0 \
- --max-cycles 5 # Stop after 5 cycles (remove for unlimited)
- ```
- ### Step 4: Watch What Happens
- The agent will now do everything automatically. Here's what each cycle looks like:
- ```
- === Cycle 1 ===
- [THINK] Reading PROJECT_BRIEF.md...
- Goal: ResNet-50 on CIFAR-100, target 80%+
- No previous experiments. Starting with baseline.
- Plan: Basic ResNet-50, lr=0.1, SGD with momentum, 100 epochs.
- [EXECUTE] Creating train.py...
- Creating config.yaml...
- Running dry-run (2 steps)... โ No errors
- Launching training: nohup python train.py --config config.yaml
- PID: 12345, Log: logs/exp001.log
- [MONITOR] Training in progress... (zero LLM cost)
- 15:00 โ PID alive, GPU 98%, Epoch 12/100, loss=2.34
- 15:15 โ PID alive, GPU 97%, Epoch 25/100, loss=1.87
- 15:30 โ PID alive, GPU 98%, Epoch 38/100, loss=1.54
- ...
- 17:45 โ PID alive, GPU 97%, Epoch 100/100, loss=0.82
- 18:00 โ PID terminated. Training complete.
- [REFLECT] Parsing logs... test accuracy = 76.3%
- Result: 76.3% โ below 80% target
- Brief says: "If < 75%, try cosine annealing"
- 76.3% > 75%, so try augmentation instead.
- Decision: Add mixup augmentation, keep lr=0.1 + cosine
- Milestone logged: "Exp001: ResNet-50 baseline, 76.3%"
- === Cycle 2 ===
- [THINK] Best so far: 76.3% (Exp001)
- Plan: Add mixup (alpha=0.2) + cosine annealing schedule
- ...
- ```
- ### Step 5: Check Progress Anytime
- While the agent is running, you can check on it:
- ```bash
- # In Claude Code:
- /experiment-status --project ~/my-first-experiment
- # Or check GPU usage:
- /gpu-monitor
- ```
- You'll see something like:
- ```
- # Experiment Status โ my-first-experiment
- ## Goal
- ResNet-50 on CIFAR-100 โ 80%+ accuracy
- ## Progress
- - Cycles completed: 3
- - Current best: 79.1% (Exp003: ResNet-50 + mixup + cosine)
- - Status: TRAINING (PID 12389, GPU 0, running 1.5h)
- ## Key Results
- [04-07 15:00] Exp001: ResNet-50 baseline, 76.3%
- [04-07 18:30] Exp002: + cosine annealing, 77.8%
- [04-07 22:00] Exp003: + mixup ฮฑ=0.2, 79.1% โ best
- ## Current Training
- Epoch 67/100 | loss: 0.71 | acc: 79.4%
- ```
- ### Step 5.5: Save Progress to Obsidian or Local Text
- Enable progress export in your project `config.yaml`:
- ```yaml
- obsidian:
- enabled: true
- vault_path: "~/Documents/MyObsidianVault" # Optional
- project_subdir: "DeepResearcher/{project_name}"
- auto_append_daily: true
- ```
- If `vault_path` is set, the agent writes:
- ```text
- DeepResearcher/my-first-experiment/Dashboard.md
- DeepResearcher/my-first-experiment/Daily/YYYY-MM-DD.md
- ```
- If `vault_path` is empty, it falls back to project-local files:
- ```text
- workspace/progress_tracking/Dashboard.txt
- workspace/progress_tracking/Daily/YYYY-MM-DD.txt
- ```
- Manual refresh:
- ```bash
- /obsidian-sync --project ~/my-first-experiment
- # or
- python -m core.obsidian --project ~/my-first-experiment
- ```
- ### Step 6: Intervene If Needed
- Want to change direction? Three ways, from anywhere:
- ```bash
- # Way 1: Drop a directive file (agent reads it next cycle)
- echo "Stop trying ResNet. Switch to ViT-B/16, start with lr=1e-3" \
- > ~/my-first-experiment/workspace/HUMAN_DIRECTIVE.md
- # Way 2: Command-line flag
- python -m core.loop --project ~/my-first-experiment \
- --directive "Try label smoothing 0.1"
- # Way 3: Edit memory directly (for permanent changes)
- vim ~/my-first-experiment/workspace/MEMORY_LOG.md
- ```
- ## Human-in-the-Loop Playbook
- Use the agent as an operator, not a replacement researcher.
- ```text
- Human decides:
- - goal
- - constraints
- - forbidden directions
- - when to pivot
- Agent executes:
- - code edits
- - runs
- - monitoring
- - summaries
- ```
- Write stable rules in `PROJECT_BRIEF.md`, and temporary steering in `HUMAN_DIRECTIVE.md`.
- ```md
- # HUMAN_DIRECTIVE.md
- - Do not change the dataset.
- - Try label smoothing 0.1 before changing the backbone.
- - Stop this direction if gain stays below 0.3 for 3 runs.
- - Compare against the last trusted baseline, not just the latest run.
- ```
- Case 1: Safer ablation
- ```md
- - Only change augmentation.
- - Keep model, optimizer, and training budget fixed.
- - Report a clean comparison table after each run.
- ```
- Case 2: Deliberate pivot
- ```md
- - Current ResNet line is saturated.
- - Switch to ViT-B/16 only if the last 3 runs plateau.
- - Before switching, write a short rationale.
- ```
- Case 3: Suspicious result
- ```md
- - Accuracy jumped unexpectedly.
- - Re-run with the same seed and one new seed.
- - Do not claim improvement until both runs reproduce.
- ```
- Rule of thumb: let the agent handle repetition, but keep direction, interpretation, and responsibility human.
- ### Step 7: Mobile Monitoring with [Happy Coder](https://github.com/slopus/happy) (Optional)
- Want to check experiments from your phone? Install [Happy Coder](https://happy.engineering/) ([iOS](https://apps.apple.com/us/app/happy-codex-claude-code-app/id6748571505) / [Android](https://play.google.com/store/apps/details?id=com.ex3ndr.happy)):
- ```bash
- # Install CLI (one time)
- npm install -g happy-coder
- # Start session through Happy instead of claude
- happy
- # Inside the session, launch your experiment:
- /auto-experiment --project ~/my-first-experiment --gpu 0
- ```
- Now on your phone you can:
- - Get **push notifications** when experiments finish or the agent needs input
- - **Check results** while commuting
- - **Send directives** ("try learning rate 1e-5") from anywhere
- - **Switch between phone and desktop** seamlessly
- - All communication is **end-to-end encrypted**
- ```
- โโโโโโโโโโโโ encrypted โโโโโโโโโโโโ
- โ Desktop โ โโโโโโโโโโโโโโโโบ โ Phone โ
- โ Claude โ relay โ Happy โ
- โ Code โ โ Coder โ
- โโโโโโโโโโโโค โโโโโโโโโโโโค
- โ Agent โ โ push notify โโ โ "Try โ
- โ running โ โ lr=1e-5"โ
- โ 24/7 โ โโ status โโโโโบ โ โ Got it โ
- โโโโโโโโโโโโ โโโโโโโโโโโโ
- ```
- ### What a Good PROJECT_BRIEF.md Looks Like
- The brief is your main lever. Here are examples for different scenarios:
- <details>
- <summary><b>Example: Fine-tuning a pretrained model</b></summary>
- ```markdown
- # Goal
- Fine-tune ViT-B/16 (pretrained on ImageNet-21K) on Oxford Flowers-102.
- Target: 95%+ test accuracy.
- # Codebase
- - Training script: finetune.py (already exists)
- - Config: configs/vit_flowers.yaml
- - Data: /data/flowers102/ (already downloaded)
- - Pretrained weights: /models/vit-b16-21k.pth
- # What to Try
- 1. First: freeze backbone, train classifier head only (10 epochs, lr=1e-2)
- 2. Then: unfreeze all, fine-tune end-to-end (30 epochs, lr=1e-4)
- 3. If stuck below 93%: try layer-wise lr decay (0.65)
- 4. If above 94%: try test-time augmentation
- # Constraints
- - GPU 0, batch size 64
- - Save best checkpoint based on val accuracy
- ```
- </details>
- <details>
- <summary><b>Example: Hyperparameter search</b></summary>
- ```markdown
- # Goal
- Find the best hyperparameters for our GAN on CelebA-HQ 256x256.
- Target: FID < 15.
- # Codebase
- - train_gan.py, configs/celeba_gan.yaml
- - Data: /data/celeba_hq_256/
- - Evaluation: eval_fid.py --real_dir /data/celeba_hq_256/val
- # Search Space
- - Learning rate: [1e-4, 2e-4, 5e-4]
- - Beta1: [0.0, 0.5]
- - Discriminator steps per generator step: [1, 2, 5]
- - Spectral norm: [yes, no]
- # Strategy
- Start with lr=2e-4, beta1=0.0, d_steps=1, spectral_norm=yes (baseline).
- Change ONE variable at a time. Run each for 50K steps.
- Always evaluate FID after training.
- # Constraints
- - GPU 0-1 (can use both)
- - Max 50K steps per run (~4 hours)
- ```
- </details>
- <details>
- <summary><b>Example: Debugging a training issue</b></summary>
- ```markdown
- # Goal
- Figure out why our transformer model diverges after epoch 20.
- Currently: loss explodes from 0.5 to NaN around epoch 20-25.
- # Codebase
- - train_transformer.py, model/transformer.py
- - Config: configs/base.yaml
- - Logs from failed runs: logs/failed_run_001.log, logs/failed_run_002.log
- # What to Investigate
- 1. Check gradient norms โ add gradient clipping (max_norm=1.0)
- 2. Try lower learning rate (current: 1e-3, try: 1e-4, 5e-5)
- 3. Check if it's a specific layer โ add per-layer gradient logging
- 4. Try warmup (1000 steps) if not already present
- 5. Check data โ are there any NaN/Inf in the dataset?
- # Constraints
- - GPU 0, run each test for 30 epochs (enough to see if it diverges)
- - Log gradient norms every 100 steps
- ```
- </details>
- ### FAQ
- <details>
- <summary><b>Q: How much does it cost to run?</b></summary>
- About $0.08 per 24-hour cycle (if training takes 8 hours). The secret: zero LLM calls during training. You only pay for the THINK and REFLECT phases (~10 min each).
- </details>
- <details>
- <summary><b>Q: Can it modify my existing code?</b></summary>
- Yes. The Code Agent can read, write, and modify any file in your project. It will make changes, dry-run to verify, then launch training. It won't touch protected files (PROJECT_BRIEF.md, MEMORY_LOG.md).
- </details>
- <details>
- <summary><b>Q: What if the agent goes in a wrong direction?</b></summary>
- Drop a directive: `echo "Stop. Go back to the ResNet approach" > workspace/HUMAN_DIRECTIVE.md`. The agent reads it next cycle with highest priority.
- </details>
- <details>
- <summary><b>Q: Can I run multiple projects at the same time?</b></summary>
- Yes. Launch separate agent instances in different terminals/tmux sessions, each pointing to a different project and GPU.
- </details>
- <details>
- <summary><b>Q: What happens if training crashes?</b></summary>
- The monitor detects the process died, captures the error log, and passes it to REFLECT. The agent will analyze the crash, fix the code, and retry.
- </details>
- <details>
- <summary><b>Q: Can I use it with PyTorch / TensorFlow / JAX?</b></summary>
- Yes. The agent works with any training framework. It just launches shell commands and reads log files โ it doesn't care what framework produces them.
- </details>
- ---
- ## One-Click Install (Claude + Codex)
- All features are packaged as Claude Code slash commands and Codex local skills.
- **One command to install:**
- ```bash
- python install.py
- ```
- After installation, you get:
- - **8 slash commands** in Claude Code
- - **8 local skills** in Codex (restart Codex after install)
- ### Core Skills
- | Command | What It Does |
- |---------|-------------|
- | `/auto-experiment` | Launch the 24/7 autonomous THINKโEXECUTEโREFLECT experiment loop |
- | `/experiment-status` | Check running experiments: progress, metrics, cycle count, GPU usage |
- | `/gpu-monitor` | Quick GPU status: free/busy, memory, utilization, running processes |
- ### Research Skills
- | Command | What It Does |
- |---------|-------------|
- | `/daily-papers` | Daily arXiv recommendations with automatic dedup |
- | `/paper-analyze 2312.12345` | Deep paper analysis + extract real figures from arXiv source |
- | `/conf-search --venue CVPR2025 --query "motion"` | Search CVPR/NeurIPS/ICML/ICLR/AAAI/ECCV... |
- | `/progress-report` | Generate structured progress report with metrics |
- | `/obsidian-sync` | Refresh Obsidian or local progress notes |
- ### Usage Example
- ```bash
- # Step 1: Install skills (one time)
- python install.py
- # Step 2a: In Claude Code, launch an experiment loop
- /auto-experiment --project /path/to/my_project --gpu 0
- # Step 2b: In Codex, use the matching local skill
- $auto-experiment
- # Step 3: Check how it's going
- /experiment-status --project /path/to/my_project
- # Step 4: Check GPU resources
- /gpu-monitor
- # Step 5: Read papers while the agent trains for you
- /daily-papers --topics "vision transformer, image classification"
- ```
- ### Uninstall
- ```bash
- python install.py --uninstall
- ```
- ---
- ## Supported LLM Providers
- Works with **Anthropic-compatible and OpenAI-compatible APIs** out of the box,
- and can also run on a **flat-rate subscription** instead of per-token billing
- via the local CLIs.
- | Tier | Anthropic (Claude) | OpenAI (Codex/GPT) | Best For |
- |------|-------------------|-------------------|----------|
- | **Fast** | `claude-sonnet-4-6` | `codex-5.3` | Daily experiments, iteration |
- | **Strongest** | `claude-opus-4-6` | `gpt-5.4` | Complex reasoning, architecture decisions |
- ### Authentication mode: API key vs. subscription
- | Mode | `provider` value | Billing | Requires | Tool-use support |
- |------|------------------|---------|----------|------------------|
- | API โ Anthropic-compatible | `anthropic` | Per-token, via `ANTHROPIC_API_KEY` or custom env | `pip install anthropic` | โ Full |
- | API โ OpenAI-compatible | `openai` | Per-token, via `OPENAI_API_KEY` or custom env | `pip install openai` | โ Full |
- | **Subscription โ Claude** | `claude_cli` | Flat-rate, uses your Claude Code / Pro / Max plan | `claude` CLI installed and logged in | โ Full |
- | **Subscription โ ChatGPT** | `codex_cli` | Flat-rate, uses your ChatGPT Plus / Pro plan | `codex` CLI installed and logged in | โ ๏ธ Leader only |
- Tool execution is driven by a text-based `<tool_call>` protocol injected
- into the worker's system prompt. All three "Full" providers can be forced
- into pure text-oracle mode so they honor the protocol (for `claude_cli`
- the framework passes `--tools ""` to disable built-in CLI tools). The
- `codex` CLI currently offers no equivalent flag โ its internal agentic
- loop will bypass the protocol and the framework cannot recover PIDs from
- experiments it launches. Use `codex_cli` only for the leader/think path
- where no tools are needed.
- Switch provider in `config.yaml`:
- ```yaml
- agent:
- # Pay-per-token (needs API key):
- provider: "anthropic" # or "openai"
- model: "claude-sonnet-4-6" # or "codex-5.3"
- base_url: "" # optional compatible endpoint override
- api_key_env: "" # optional custom key env var name
- auth_token_env: "" # optional custom bearer token env var
- # Flat-rate subscription (needs CLI login instead of API key):
- # provider: "claude_cli" # or "codex_cli"
- ```
- Compatible API examples
- (illustrative only in this repo โ these endpoint/model combinations have not
- been live-smoke-tested here):
- ```yaml
- # Qwen / DashScope
- agent:
- provider: "openai"
- model: "qwen-plus"
- base_url: "https://dashscope.aliyuncs.com/compatible-mode/v1"
- api_key_env: "DASHSCOPE_API_KEY"
- # GLM / BigModel
- agent:
- provider: "openai"
- model: "glm-4.5"
- base_url: "https://open.bigmodel.cn/api/paas/v4"
- api_key_env: "ZHIPUAI_API_KEY"
- # MiniMax via OpenAI-compatible endpoint
- agent:
- provider: "openai"
- model: "MiniMax-M1"
- base_url: "https://api.minimaxi.com/v1"
- api_key_env: "MINIMAX_API_KEY"
- ```
- Or set via environment (API-key modes only):
- ```bash
- # For Anthropic-compatible provider:
- export ANTHROPIC_API_KEY="sk-ant-xxxxx"
- export ANTHROPIC_BASE_URL="https://your-anthropic-compatible-endpoint"
- # For OpenAI-compatible provider:
- export OPENAI_API_KEY="sk-xxxxx"
- export OPENAI_BASE_URL="https://your-openai-compatible-endpoint/v1"
- # For subscription providers (claude_cli / codex_cli): no env var โ just
- # install the CLI once and run `claude` or `codex login` to sign in.
- ```
- ---
- ## Configuration
- By default, everything runs locally inside `project.workspace`. If you want
- to keep the controller on your laptop but run code, training, logs, and GPU
- checks on one remote server, enable the optional SSH execution mode:
- ```yaml
- execution:
- mode: "ssh"
- ssh_host: "user@your-server"
- remote_workspace: "/home/user/my_project/workspace"
- remote_python: "python3"
- ssh_args: [] # optional, e.g. ["-p", "2222"]
- ```
- In SSH mode, controller state still stays local:
- - `PROJECT_BRIEF.md`
- - `workspace/MEMORY_LOG.md`
- - `workspace/state.json`
- - `workspace/HUMAN_DIRECTIVE.md`
- - local progress / Obsidian exports
- The remote host only handles the tool-visible workspace, training process,
- training logs, PID checks, and `nvidia-smi`.
- On a **Slurm cluster**, set `mode: "slurm"`. The controller still stays on your
- laptop; training is submitted to the login node with `sbatch --parsable` over a
- single transient SSH call that exits immediately (no process is left running on
- the login node), and `sacct` is the sole liveness authority โ Slurm enforces
- `--time`, so a job is always reaped by its time limit plus a safety buffer:
- ```yaml
- execution:
- mode: "slurm"
- ssh_host: "user@login-node"
- remote_workspace: "/nfs/home/user/my_project/workspace"
- slurm_partition: "gpu-h200" # required
- slurm_time: "24:00:00" # required (--time wall limit)
- slurm_gpus_per_job: 1 # -> --gres=gpu:N
- slurm_setup: "module load cuda/12.4" # optional shell line prepended to the job
- ```
- In Slurm mode the `gpu` argument to `launch_experiment` is ignored โ Slurm
- assigns GPUs via `--gres`, so `CUDA_VISIBLE_DEVICES` is not pinned by the agent.
- See `config.yaml` for the full set of `slurm_*` options.
- ```yaml
- # config.yaml
- project:
- name: "my-research"
- brief: "PROJECT_BRIEF.md"
- execution:
- mode: "local" # or "ssh" / "slurm"
- ssh_host: "" # required in ssh/slurm mode
- remote_workspace: "" # required in ssh/slurm mode
- remote_python: "python3"
- ssh_args: []
- agent:
- provider: "anthropic" # "anthropic" or "openai"
- model: "claude-sonnet-4-6" # See model table above
- base_url: "" # Optional compatible API endpoint override
- api_key_env: "" # Optional custom API key env var
- auth_token_env: "" # Optional custom bearer token env var
- max_cycles: -1 # -1 = run forever
- max_steps_per_cycle: 3 # Max worker dispatches per cycle
- cooldown_interval: 300 # Smart cooldown polling (seconds)
- memory:
- brief_max_chars: 3000 # Tier 1 cap
- log_max_chars: 2000 # Tier 2 cap
- milestone_max_chars: 1200 # Key results cap
- max_recent_entries: 15 # Rolling decision count
- gpu:
- auto_detect: true
- reserve_last: true # Reserve last GPU for keep-alive
- monitor:
- poll_interval: 900 # Check every 15 min during training
- zero_llm: true # No LLM during monitoring
- experiment:
- mandatory_dry_run: true # Always dry-run before real training
- max_parallel: 1 # Concurrent experiments
- ```
- ---
- ## How It Compares
- | | Deep Researcher Agent | [Claude Scholar](https://github.com/Galaxy-Dawn/claude-scholar) | [AI Scientist](https://github.com/SakanaAI/AI-Scientist) | [OpenHands](https://github.com/All-Hands-AI/OpenHands) | [SWE-Agent](https://github.com/princeton-nlp/SWE-agent) |
- |--|:--:|:--:|:--:|:--:|:--:|
- | **Runs experiments autonomously** | โ | โ | โ | โ | โ |
- | **Zero-cost training monitoring** | โ | โ | โ | โ | โ |
- | **GPU management** | โ | โ | โ | โ | โ |
- | **24/7 continuous operation** | โ | โ | โ | โ | โ |
- | **Constant-size memory** | โ | โ | โ | โ | โ |
- | Paper writing | Basic | โ | โ | โ | โ |
- | Knowledge management | Basic | โ | โ | โ | โ |
- | General coding | โ | โ | โ | โ | โ |
- **Deep Researcher Agent is the only framework built for _running_ deep learning research, not just writing about it.**
- ---
- ## Project Structure
- ```
- auto-deep-researcher-24x7/
- โโโ core/ # Autonomous experiment loop engine
- โ โโโ loop.py # THINK โ EXECUTE โ REFLECT cycle
- โ โโโ execution.py # Local / SSH execution backends
- โ โโโ memory.py # Two-Tier constant-size memory
- โ โโโ monitor.py # Zero-LLM experiment monitoring
- โ โโโ agents.py # Leader-Worker agent dispatch
- โ โโโ tools.py # Minimal per-agent tool registry
- โโโ skills/ # Source skills for Claude slash commands + Codex local skills
- โ โโโ auto-experiment/ # 24/7 autonomous experiment loop
- โ โโโ experiment-status/ # Check experiment progress
- โ โโโ gpu-monitor/ # GPU status & availability
- โ โโโ daily-papers/ # Daily arXiv recommendations
- โ โโโ paper-analyze/ # Deep paper analysis + figure extraction
- โ โโโ conf-search/ # Conference paper search
- โ โโโ progress-report/ # Progress report generation
- โโโ agents/ # Agent prompt definitions
- โ โโโ leader.md # Central decision-maker
- โ โโโ idea_agent.md # Literature & hypothesis
- โ โโโ code_agent.md # Experiment execution
- โ โโโ writing_agent.md # Reporting & writing
- โโโ gpu/ # GPU utilities
- โ โโโ detect.py # Detection & monitoring
- โ โโโ keeper.py # Cloud instance keep-alive
- โโโ examples/ # Ready-to-run demos
- โโโ docs/ # Docs + translations (CN/JP)
- โโโ install.py # Claude + Codex skill installer
- โโโ config.yaml # Default configuration
- โโโ requirements.txt # Dependencies
- ```
- ---
- ## Contributing
- Areas where we'd love help:
- - More cloud GPU platforms (AWS, GCP, Lambda Labs, RunPod)
- - Experiment tracker integration (W&B, MLflow, TensorBoard)
- - New research skills (visualization, result comparison)
- - Metric extraction for more training frameworks
- See [CONTRIBUTING.md](CONTRIBUTING.md).
- ---
- ## Citation
- If you find this work useful, please cite our paper:
- ```bibtex
- @article{zhang2026autodeepresearcher,
- title={Deep Researcher Agent: Autonomous Deep Learning Experiment Framework},
- author={Zhang, Xiangyue},
- journal={arXiv preprint arXiv:2604.05854},
- year={2026},
- url={https://arxiv.org/abs/2604.05854}
- }
- ```
- Or cite the software release:
- ```bibtex
- @software{auto_deep_researcher_24x7,
- title={Deep Researcher Agent: Autonomous Deep Learning Experiment Framework},
- author={Xiangyue Zhang},
- year={2026},
- url={https://github.com/Xiangyue-Zhang/auto-deep-researcher-24x7}
- }
- ```
- ## Star History
- [](https://www.star-history.com/#Xiangyue-Zhang/auto-deep-researcher-24x7&Date)
- ## License
- Apache 2.0 โ see [LICENSE](LICENSE).
- ---
- <p align="center">
- <strong><i>"Experiments run through the night. Results arrive at dawn."</i></strong>
- </p>
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If itโs helped, pass it on.