Skip to content
Back to search

SkillZero

[EMNLP 2026] Official code for "SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization"

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/zju-real--skillzero/badge.svg)](/listings/zju-real--skillzero)

HTML

<a href="/listings/zju-real--skillzero"><img src="/api/listings/zju-real--skillzero/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Python environment

conda create -n skillzero python=3.12 -y
conda activate skillzero

pip install vllm==0.10.0
pip install flash-attn==2.7.4.post1 --no-build-isolation --no-cache-dir
pip install -e .

Log in to Weights & Biases if you use WandB logging (scripts pass `trainer.logger=['console','wandb']` in many cases):

export WANDB_API_KEY=your_key_here

Install Supported Environments

1. ALFWorld

Install with pip:

pip3 install gymnasium==0.29.1
pip3 install stable-baselines3==2.6.0
pip3 install alfworld

Download PDDL & Game files and pre-trained MaskRCNN detector (will be stored in `~/.cache/alfworld/`):

alfworld-download -f

2. Search

cd ./agent_system/environments/env_package/search/third_party
pip install -e .
pip install gym==0.26.2

Prepare dataset (data will be saved at `~/data/searchR1_processed_direct`):

cd repo_root/
python examples/data_preprocess/preprocess_search_r1_dataset.py

Build Retriever environments:

conda create -n retriever python=3.10 -y
conda activate retriever

conda install numpy==1.26.4 
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124

pip install transformers datasets pyserini huggingface_hub
conda install faiss-gpu==1.8.0 -c pytorch -c nvidia -y
pip install uvicorn fastapi

Download the index:

conda activate retriever

local_dir=~/data/searchR1
python examples/search/searchr1_download.py --local_dir $local_dir
cat $local_dir/part_* > $local_dir/e5_Flat.index
gzip -d $local_dir/wiki-18.jsonl.gz

Start the local flat e5 retrieval server:

conda activate retriever

# redirect the output to a file to avoid cluttering the terminal
# we have observed outputting to the terminal causing spikes in server response times
bash examples/search/retriever/retrieval_launch.sh > retrieval_server.log 

Validation parquet for SkillZero Search

python -m examples.data_preprocess.generate_search_r1_val

Training

All scripts live under `scripts/` and assume the repo root as working directory (they `cd` there automatically). You can run either:

bash scripts/train_alfworld_skillzero_3b.sh
bash scripts/train_search_skillzero_3b

### Merge checkpoints

See `scripts/model_merger.py` for FSDP/Megatron merge examples using paths under `./checkpoints/...`.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 192 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <h1 align="center">
  2. SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization
  3. </h1>
  4. <div align='center' style="font-size:18px;">
  5. <p>
  6. <a href="https://arxiv.org/abs/2604.02268">
  7. <img src="https://img.shields.io/badge/Paper-arxiv%3A2604.02268-blue" alt="Paper"/>
  8. </a>
  9. <a href="https://huggingface.co/papers/2604.02268">
  10. <img src="https://img.shields.io/badge/Daily%20Paper-huggingface-yellow" alt="HF Paper"/>
  11. </a>
  12. </p>
  13. </div>
  14. ## 🔥 Overview
  15. We introduce **SKILL0**, an in-context reinforcement learning framework designed for *skill internalization*.
  16. <div align="center" style="display:flex; justify-content:center; gap:20px; align-items:flex-start;">
  17. <img src="docs/skillzero/motivation.png" alt="motivation" style="width:40%;">
  18. <img src="docs/skillzero/method.png" alt="method" style="width:58%;">
  19. </div>
  20. SKILL0 achieves substantial improvements over the standard RL baseline on ALFWorld and Search-QA.
  21. <div align="center">
  22. <img src="docs/skillzero/metric.png" alt="Logo" style="width:80%;">
  23. </div>
  24. ## 🗞️ News
  25. - **`2026-8-6`**: 🔥🔥 We released [AgentOPSD](https://github.com/ZethWang/AgentOPSD), introducing **recursive credit update** for SDAR.
  26. - **`2026-7-29`**: 🔥🔥 We released [SkillRise](https://github.com/Within-yao/SkillRise), introducing **cross-task skill evolution** via agentic RL.
  27. - **`2026-7-17`**: 🔥🔥 We released [SEED](https://github.com/jinyangwu/SEED), introducing **self-evolving** opd beyond skill internalization.
  28. - **`2026-6-25`**: 🔥 We released [OPID](https://github.com/jinyangwu/OPID), introducing **skill evolving** beyond skill internalization.
  29. - **`2026-5-15`**: 🔥 Our new work was released: [SDAR](https://github.com/ZJU-REAL/SDAR), which introduces Self-Distilled Agentic Reinforcement Learning for skill internalization.
  30. - **`2026-5-07`**: 🔥 Our new work was released: [SKILL1](https://github.com/AlphaLab-USTC/Skill1), which evloves skill-augmented agents in **one** unified policy.
  31. - **`2026-4-03`**: We release our paper and code.
  32. ## 🛠️ Installation
  33. ### Python environment
  34. ```bash
  35. conda create -n skillzero python=3.12 -y
  36. conda activate skillzero
  37. pip install vllm==0.10.0
  38. pip install flash-attn==2.7.4.post1 --no-build-isolation --no-cache-dir
  39. pip install -e .
  40. ```
  41. Log in to Weights & Biases if you use WandB logging (scripts pass `trainer.logger=['console','wandb']` in many cases):
  42. ```bash
  43. export WANDB_API_KEY=your_key_here
  44. ```
  45. ### Install Supported Environments
  46. #### 1. ALFWorld
  47. Install with pip:
  48. ```bash
  49. pip3 install gymnasium==0.29.1
  50. pip3 install stable-baselines3==2.6.0
  51. pip3 install alfworld
  52. ```
  53. Download PDDL & Game files and pre-trained MaskRCNN detector (will be stored in `~/.cache/alfworld/`):
  54. ```bash
  55. alfworld-download -f
  56. ```
  57. #### 2. Search
  58. ```bash
  59. cd ./agent_system/environments/env_package/search/third_party
  60. pip install -e .
  61. pip install gym==0.26.2
  62. ```
  63. Prepare dataset (data will be saved at `~/data/searchR1_processed_direct`):
  64. ```bash
  65. cd repo_root/
  66. python examples/data_preprocess/preprocess_search_r1_dataset.py
  67. ```
  68. Build Retriever environments:
  69. ```bash
  70. conda create -n retriever python=3.10 -y
  71. conda activate retriever
  72. conda install numpy==1.26.4
  73. pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124
  74. pip install transformers datasets pyserini huggingface_hub
  75. conda install faiss-gpu==1.8.0 -c pytorch -c nvidia -y
  76. pip install uvicorn fastapi
  77. ```
  78. Download the index:
  79. ```bash
  80. conda activate retriever
  81. local_dir=~/data/searchR1
  82. python examples/search/searchr1_download.py --local_dir $local_dir
  83. cat $local_dir/part_* > $local_dir/e5_Flat.index
  84. gzip -d $local_dir/wiki-18.jsonl.gz
  85. ```
  86. Start the local flat e5 retrieval server:
  87. ```bash
  88. conda activate retriever
  89. # redirect the output to a file to avoid cluttering the terminal
  90. # we have observed outputting to the terminal causing spikes in server response times
  91. bash examples/search/retriever/retrieval_launch.sh > retrieval_server.log
  92. ```
  93. Validation parquet for SkillZero Search
  94. ```bash
  95. python -m examples.data_preprocess.generate_search_r1_val
  96. ```
  97. ### Training
  98. All scripts live under `scripts/` and assume the repo root as working directory (they `cd` there automatically). You can run either:
  99. ```bash
  100. bash scripts/train_alfworld_skillzero_3b.sh
  101. bash scripts/train_search_skillzero_3b
  102. ### Merge checkpoints
  103. See `scripts/model_merger.py` for FSDP/Megatron merge examples using paths under `./checkpoints/...`.
  104. ```
  105. ## ⭐️ Citation
  106. If you find this project useful, welcome to cite us.
  107. ```bit
  108. @article{lu2026skill0,
  109. title={Skill0: In-context agentic reinforcement learning for skill internalization},
  110. author={Lu, Zhengxi and Yao, Zhiyuan and Wu, Jinyang and Han, Chengcheng and Gu, Qi and Cai, Xunliang and Lu, Weiming and Xiao, Jun and Zhuang, Yueting and Shen, Yongliang},
  111. journal={arXiv preprint arXiv:2604.02268},
  112. year={2026}
  113. }
  114. @article{lu2026sdar,
  115. title={Self-distilled agentic reinforcement learning},
  116. author={Lu, Zhengxi and Yao, Zhiyuan and Han, Zhuowen and Wang, Zi-Han and Wu, Jinyang and Gu, Qi and Cai, Xunliang and Lu, Weiming and Xiao, Jun and Zhuang, Yueting and others},
  117. journal={arXiv preprint arXiv:2605.15155},
  118. year={2026}
  119. }
  120. @article{shi2026skill1,
  121. title={Skill1: Unified evolution of skill-augmented agents via reinforcement learning},
  122. author={Shi, Yaorui and Chen, Yuxin and Lu, Zhengxi and Miao, Yuchun and Liu, Shugui and Gu, Qi and Cai, Xunliang and Wang, Xiang and Zhang, An},
  123. journal={arXiv preprint arXiv:2605.06130},
  124. year={2026}
  125. }
  126. @article{wu2026seed,
  127. title={SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning},
  128. author={Wu, Jinyang and Yang, Shuo and Lu, Zhengxi and Zhang, Fan and Shen, Yuhao and Feng, Lang and Luo, Haoran and Lian, Zheng and Zhang, Shuai and Wen, Zhengqi and others},
  129. journal={arXiv preprint arXiv:2607.14777},
  130. year={2026}
  131. }
  132. @article{yang2026opid,
  133. title={Opid: On-policy skill distillation for agentic reinforcement learning},
  134. author={Yang, Shuo and Wu, Jinyang and Lu, Zhengxi and Shen, Yuhao and Zhang, Fan and Feng, Lang and Zhang, Shuai and Luo, Haoran and Lian, Zheng and Wen, Zhengqi and others},
  135. journal={arXiv preprint arXiv:2606.26790},
  136. year={2026}
  137. }
  138. @article{yao2026skillrise,
  139. title={SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution},
  140. author={Yao, Zhiyuan and Chen, Yuxin and Lu, Zhengxi and Xu, Zishan and Sun, Yueqing and Guo, Yifu and Lu, Yuquan and Cai, Zhengzhou and Zhang, Kangning and Han, Zhuowen and others},
  141. journal={arXiv preprint arXiv:2607.26784},
  142. year={2026}
  143. }
  144. @article{wang2026agentopsd,
  145. title={AgentOPSD: Recursive Self-Distillation for Agentic Reinforcement Learning},
  146. author={Wang, Zi-Han and Lu, Zhengxi and Yao, Zhiyuan and Wu, Jinyang and Wu, Jie and Cai, Zhengzhou and Sun, Yueqing and Ye, Ziang and Hao, Linji and Gu, Qi and others},
  147. journal={arXiv preprint arXiv:2608.05987},
  148. year={2026}
  149. }
  150. ```
  151. ## 🤝 Acknowledgement
  152. This project builds on [AgentOCR](https://github.com/langfengQ/AgentOCR), [verl-agent](https://github.com/langfengQ/verl-agent), [veRL](https://github.com/volcengine/verl), [ALFWorld](https://github.com/alfworld/alfworld), [SkillRL](https://github.com/aiming-lab/SkillRL), and [Search-R1](https://github.com/PeterGriffinJin/Search-R1). We thank the authors of those projects.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.