Skip to content
Back to search

MemSkill

[NeurIPS'26] MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/viktoraxelsen--memskill/badge.svg)](/listings/viktoraxelsen--memskill)

HTML

<a href="/listings/viktoraxelsen--memskill"><img src="/api/listings/viktoraxelsen--memskill/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

# Clone the repository
git clone https://github.com/ViktorAxelsen/MemSkill
cd MemSkill

# Create and activate virtual environment
conda create -n memskill python=3.10
conda activate memskill

# vllm
pip install vllm==0.6.3
# PyTorch
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124
# Flash-Attn (or you can specify `--disable-flash-attn` in the .sh scripts to disable it)
pip install flash-attn --no-build-isolation
# Others
pip install -r requirements.txt

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 363 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <div align="center">
  2. <img src="assets/logo.png" alt="LLMRouter Logo" width="300">
  3. </div>
  4. <h1 align="center">MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents</h1>
  5. <div align="center">
  6. <p>
  7. <a href='https://viktoraxelsen.github.io/MemSkill/'><img src='https://img.shields.io/badge/Project-Page-00d9ff?style=for-the-badge&logo=github&logoColor=white'></a>
  8. <a href='https://arxiv.org/abs/2602.02474'><img src='https://img.shields.io/badge/arXiv-2602.02474-ff6b6b?style=for-the-badge&logo=arxiv&logoColor=white'></a>
  9. <a href="https://huggingface.co/papers/2602.02474"><img src="https://img.shields.io/badge/HuggingFace-Paper-FFD21E?style=for-the-badge&logo=huggingface&logoColor=FFD21E" alt="HuggingFace Paper"></a>
  10. <a href="https://huggingface.co/collections/XaiverZ/memskill"><img src="https://img.shields.io/badge/HuggingFace-Collection-FFD21E?style=for-the-badge&logo=huggingface&logoColor=FFD21E" alt="HuggingFace Collection"></a>
  11. <br>
  12. <a href="https://github.com/ViktorAxelsen/MemSkill/stargazers"><img src='https://img.shields.io/github/stars/ViktorAxelsen/MemSkill?color=f1e05a&style=for-the-badge&logo=star&logoColor=white' /></a>
  13. <a href="https://github.com/ViktorAxelsen/MemSkill/forks"><img src='https://img.shields.io/github/forks/ViktorAxelsen/MemSkill?color=2ea44f&style=for-the-badge&logo=git&logoColor=white' /></a>
  14. <a href="https://github.com/ViktorAxelsen/MemSkill/issues"><img src='https://img.shields.io/github/issues/ViktorAxelsen/MemSkill?color=d73a49&style=for-the-badge&logo=github&logoColor=white' /></a>
  15. <a href="https://deepwiki.com/ViktorAxelsen/MemSkill"><img src="https://img.shields.io/badge/DeepWiki-MemSkill-6B4FBB?style=for-the-badge&logo=readthedocs&logoColor=white" alt="DeepWiki"></a>
  16. <!-- <a href="https://www.python.org/downloads/release/python-3109/"><img src="https://img.shields.io/badge/PYTHON-3.10-3776AB?style=for-the-badge&logo=python&logoColor=white" alt="Python"></a> -->
  17. <!-- <a href="x" style="text-decoration:none;"><img src="https://img.shields.io/badge/TWITTER-ANNOUNCEMENTS-1DA1F2?style=for-the-badge&logo=x&logoColor=white" alt="Twitter"></a> -->
  18. <a href="LICENSE"><img src="https://img.shields.io/badge/LICENSE-Apache-2EA44F?style=for-the-badge" alt="License"></a>
  19. </p>
  20. </div>
  21. ## 🧩 Overview
  22. **MemSkill** is a framework for learning and evolving **memory skills** for long-horizon agents. It replaces static, hand-designed memory operations with a data-driven loop where skills are **learned, refined, and reused** from task feedback, enabling more adaptive memory construction across settings.
  23. ❗The skills evolved by **MemSkill** are **NOT** experiential/procedural memory/insights themselves. Rather, they are a form of **meta-memory** that focuses on **what kinds of memory to extract, how to remember and where to focus, and what to preserve or forget**. This is also why we call them **memory skills**: they capture the way or skill of remembering, rather than the remembered content itself.
  24. **Highlights**
  25. - **Skill-conditioned memory construction**: Compose a small set of relevant skills for each span and construct memories in one pass.
  26. - **Skill evolution from hard cases**: Periodically mine challenging examples to refine existing skills and propose new ones.
  27. - **Reusable skill bank**: Maintain a shared, evolving skill bank that supports transfer across datasets and base models.
  28. - **High-throughput evaluation**: Multi-API-key round-robin for stable, parallel calls.
  29. - **Scalable training and runs**: Multi-threading and multi-processing for large-scale training and evaluation.
  30. <div align="center">
  31. <img src="./assets/model.png" width="800" alt="MemSkill">
  32. </div>
  33. ## 📰 News
  34. - 🎉 **[2026-09]: MemSkill was accepted by NeurIPS'26!**
  35. - 🚀 **[2026-03]**: `--locomo-train-query-sampling-ratio` is now available for training-time stratified test-query sampling on LoCoMo. It significantly reduces evaluation cost during training by sampling LoCoMo test queries by category, while leaving the full-evaluation protocol unchanged for `eval-only` and formal testing. For more details, please refer to [Commonly Used Configs](#️-commonly-used-configs).
  36. - 🛠️ **[2026-03]**: We have added support for interrupted training recovery. You can now resume training in the `train_*.sh` scripts by passing `--load-checkpoint`, which restores key training state such as the controller/optimizer, operation bank, designer state (for example, the rolling failure-case pool), and other resume-critical metadata. At the moment, recovery is supported only from checkpoints saved at **outer-epoch boundaries**. By default, resumed runs continue logging to the original W&B run; if you prefer a fresh run for logging, use `--resume-new-wandb-run` instead. For more details, please refer to [Commonly Used Configs](#️-commonly-used-configs).
  37. - 🚀 **[2026-03]**: We have improved the parallel memory extraction pipeline for evaluation and cache building, making MemSkill noticeably faster in large-scale runs. We also added clearer controls for concurrency with `--inference-workers` at the sample level and `--inference-session-workers` within each sample at the chunk/span level, which together can significantly accelerate memory extraction. For more details, please refer to [Commonly Used Configs](#️-commonly-used-configs).
  38. - ⭐ **[2026-03]**: We have released the MemSkill controller weights in our [Hugging Face collection](https://huggingface.co/collections/XaiverZ/memskill), which can now be used directly for inference on suitable datasets. Please note that differences in experimental environments and settings may require some adaptation; when necessary, we recommend retraining and tuning key hyperparameters on a held-out validation split, especially `chunk_size` and the number of selected skills during inference (`action_top_k`), to ensure reliable performance. We hope these resources help advance self-evolving agent memory systems, and we'd be glad to hear from the community.
  39. - 🔥 **[2026-02]**: We are honored to be featured in the 🤗 HuggingFace [#3 Paper of the day](https://huggingface.co/papers/2602.02474)
  40. - 🚀 **[2026-02]**: **MemSkill** is officially released — a new paradigm for agent memory that learns reusable skills 🔁 and evolves them from data over time 🧠, improving memory quality and generalization across long, open-ended interactions ✨.
  41. ## 🔗 Links
  42. - [Overview](#-overview)
  43. - [News](#-news)
  44. - [Get Started](#-get-started)
  45. - [Installation](#installation)
  46. - [Preparing Training Data](#-preparing-training-data)
  47. - [Experiments](#-experiments)
  48. - [Extending to New Datasets and Evaluation Protocol](#-extending-to-new-datasets-and-evaluation-protocol)
  49. - [Commonly Used Configs](#️-commonly-used-configs)
  50. - [Citation](#-citation)
  51. ## 🚀 Get Started
  52. ### Installation
  53. ```bash
  54. # Clone the repository
  55. git clone https://github.com/ViktorAxelsen/MemSkill
  56. cd MemSkill
  57. # Create and activate virtual environment
  58. conda create -n memskill python=3.10
  59. conda activate memskill
  60. # vllm
  61. pip install vllm==0.6.3
  62. # PyTorch
  63. pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124
  64. # Flash-Attn (or you can specify `--disable-flash-attn` in the .sh scripts to disable it)
  65. pip install flash-attn --no-build-isolation
  66. # Others
  67. pip install -r requirements.txt
  68. ```
  69. ### 📊 Preparing Training Data
  70. MemSkill builds training and evaluation data from the datasets below. Please download data from the official sources and place them under `data/`. Unless otherwise noted, splits are already configured in our codebase.
  71. #### **1) LoCoMo**
  72. - Download LoCoMo from the official repo: [LoCoMo](https://github.com/snap-research/locomo)
  73. - **Splits**: LoCoMo splits are **already configured in `main.py`** (no extra split file needed).
  74. - Put the downloaded files under:
  75. - `data/locomo10.json`
  76. #### **2) LongMemEval**
  77. - We use **LongMemEval-S** from: [LongMemEval](https://github.com/xiaowu0162/LongMemEval)
  78. - **Important**: LongMemEval-S is used for **transfer evaluation only**. That is, skills trained on LoCoMo are **directly evaluated** on LongMemEval-S without additional training.
  79. - Put the downloaded files under:
  80. - `data/longmemeval_s_cleaned.json`
  81. - Use our split file:
  82. - `data/longmemeval_s_splits.json` (**We use test split only**)
  83. #### **3) HotpotQA**
  84. - Download HotpotQA from: [HotpotQA-Modified](https://huggingface.co/datasets/BytedTsinghua-SIA/hotpotqa/tree/main) (Source: [HotpotQA](https://hotpotqa.github.io/))
  85. - We evaluate on three test files:
  86. - `data/eval_50.json`
  87. - `data/eval_100.json`
  88. - `data/eval_200.json`
  89. These correspond to **increasing context length**, where each query context is constructed by concatenating **50 / 100 / 200 documents** (following the long-context evaluation protocol we adopt in our experiments).
  90. #### **4) ALFWorld**
  91. Please follow the official instructions to install dependencies and download assets: [ALFWorld](https://github.com/alfworld/alfworld)
  92. We use **offline expert trajectories** as the interaction corpus for memory construction. We provide a one-command script to collect and save trajectories:
  93. ```bash
  94. # Collect expert trajectories for train / seen / unseen splits
  95. python alfworld_replay.py --split train --output ./data/alfworld_train_offline.json
  96. python alfworld_replay.py --split eval_in_distribution --output ./data/alfworld_expert_eval_in_distribution.json
  97. python alfworld_replay.py --split eval_out_of_distribution --output ./data/alfworld_expert_eval_out_of_distribution.json
  98. ```
  99. Note that:
  100. - We collect `seen` and `unseen` expert plans **only to keep data formats consistent** and make evaluation easier. They are not used for training.
  101. - The saved trajectories will be saved under `data/` by default.
  102. > **ALFWorld Training Data Preparation Workflow**
  103. We separate data into **two batches with different roles**.
  104. **Batch A: Offline expert trajectories (memory construction batch)**
  105. We first collect expert rollouts (the JSON files above). During training, we sample a batch of trajectories and:
  106. - split each trajectory trace into **contiguous spans** (processed sequentially span by span)
  107. - build an **episode specific memory bank** by running MemSkill on these spans
  108. - record the controller’s Top-K skill selections (and the associated policy info needed for RL updates)
  109. This batch is used to teach the controller how to **compose skills** for memory construction from realistic interaction traces, without requiring environment interaction.
  110. **Batch B: Environment evaluation episodes (reward batch)**
  111. To obtain task-level feedback, we sample a batch of ALFWorld tasks and:
  112. - run the agent in the environment using the memory bank produced by MemSkill in the Batch A
  113. - compute the **task signal** (e.g., success rate) as the reward feedback
  114. - log difficult failures as hard cases for the designer
  115. This batch provides the supervision signal that tells the controller whether its skill composition actually helps long-horizon execution.
  116. **How they work together**
  117. - Batch A provides the behavior data (skill choices on spans) needed to optimize the controller policy.
  118. - Batch B provides the downstream task feedback that makes the optimization meaningful.
  119. - The designer then mines hard cases from Batch B outcomes to refine existing skills and propose new ones.
  120. In short, ALFWorld uses **offline traces for scalable memory construction training (Batch A)** and **environment rollouts for task feedback and skill evolution (Batch B)**.
  121. ❗For integrating more datasets, our framework is designed to be flexible and easy to extend to new settings (different interaction formats, query styles, and evaluation protocols). See [Extending to New Datasets and Evaluation Protocol](#-extending-to-new-datasets-and-evaluation-protocol) for step-by-step instructions.
  122. ## 🧪 Experiments
  123. Before running, please check the parameter configuration in the `.sh` scripts.
  124. > [!IMPORTANT]
  125. >
  126. > **Before running, please review **[Commonly Used Configs](#️-commonly-used-configs)** and update the `.sh` scripts with your dataset paths, API settings, and model choices.**
  127. ### 🖥️ Training
  128. Choose the training script based on the dataset you want to use. Make sure `--data-file`, `--model`, and API settings are set correctly.
  129. ```bash
  130. bash train_locomo.sh
  131. # or
  132. bash train_alfworld.sh
  133. ```
  134. ### 🧭 Evaluation
  135. Use the matching evaluation script after training. Set --load-checkpoint and dataset paths before running.
  136. ```bash
  137. bash eval_locomo.sh
  138. # or
  139. bash eval_alfworld.sh
  140. # or
  141. bash eval_hp.sh
  142. # or
  143. bash eval_longmemeval.sh
  144. ```
  145. ## 🔧 Extending to New Datasets and Evaluation Protocol
  146. MemSkill is designed to be extensible across diverse datasets and evaluation protocols. To add a new dataset or evaluation protocol:
  147. - Implement a data processor in `src/data_processing/` (parse raw data, build sessions/spans, and expose QA items if applicable).
  148. - Implement an evaluator in `src/eval/` (prompt construction, answer extraction, metric computation).
  149. - Register the new processor/evaluator in `src/data_processing/__init__.py` and `src/eval/__init__.py`.
  150. - Add a run script (train/eval) with the dataset name, data paths, and retrieval settings.
  151. In practice, you only need to define how to segment interaction history into spans, how to format prompts for memory construction and QA, and how to score outputs. The controller–executor–designer loop and skill‑bank evolution remain unchanged.
  152. Note: `src/trainer.py` usually does not need changes when adding a new dataset; only interactive environments (e.g., ALFWorld‑style) or custom training loops require trainer modifications.
  153. ## ⚙️ Commonly Used Configs
  154. These are the parameters most frequently used in the training/eval `.sh` scripts:
  155. **Core run settings**
  156. - `--dataset`: dataset name (`locomo`, `longmemeval`, `hotpotqa`, `alfworld`)
  157. - `--data-file`: path to the main dataset file
  158. - `--model`: base LLM name
  159. - `--designer-model`: base LLM name for designer (default: the same as --model)
  160. - `--api`: use API-based inference (without this flag, the model is loaded locally with vLLM for inference)
  161. - `--api-base`: API endpoint
  162. - `--api-key`: one or more API keys
  163. - `--disable-flash-attn`: Flash Attention is enabled by default; add this flag to disable it.
  164. > [!NOTE]
  165. >
  166. > To reduce the training cost introduced by open-ended skill evolution, one practical option is to use a smaller `--model` together with a larger `--designer-model`. In our experiments, we can still observe an increasing reward curve even when both `--model` and `--designer-model` are small. Generally speaking, however, a larger `--designer-model` makes skill evolution more effective while adding almost no extra training cost, because the designer contributes only a negligible number of LLM calls and token comsumption compared to the controller.
  167. >
  168. > Another option is to reduce the number of retrieved memories during training, namely `--mem-top-k`, since the dominant training cost usually comes from evaluating test queries. We also provide `--locomo-train-query-sampling-ratio`, a query-sampling option that can stratify test queries to reduce the evaluation burden during training while preserving stability and effectiveness. We hope these practical insights and experiences help the community follow, reproduce, and adapt MemSkill more easily.
  169. **Retrieval & memory**
  170. - `--retriever`: retriever type (`contriever`, `dpr`, `dragon`)
  171. - `--mem-top-k`: top‑K memories for training
  172. - `--mem-top-k-eval`: top‑K memories for evaluation
  173. - `--locomo-train-query-sampling-ratio`: LoCoMo-only training-time query sampling ratio for stratified test-query evaluation (`1.0` keeps full evaluation)
  174. - `--session-mode`: span granularity (`turn`, `turn-pair`, `full-session`, `fixed-length`)
  175. - `--chunk-size`, `--chunk-overlap`: fixed-length chunking size/overlap
  176. - `--inference-workers`: number of parallel workers for sample-level memory inference(`1` = serial)
  177. - `--inference-session-workers`: number of parallel workers for session/span-level memory inference within each sample (`1` = serial; **values > 1 may degrade performance**)
  178. > [!WARNING]
  179. >
  180. > `--inference-workers` increases sample-level concurrency and can usually be set higher as long as your API rate limit can sustain it, without affecting model performance. `--inference-session-workers` increases within-sample concurrency for sequential memory extraction and can speed up runs, but values greater than `1` may hurt task performance. Together with `--chunk-size`, these knobs can significantly accelerate memory extraction, so tune them carefully against quality.
  181. **Training**
  182. - `--batch-size`: episodes per PPO update
  183. - `--encode-batch-size`: batch size for embedding encoder
  184. - `--inner-epochs`, `--outer-epochs`: training schedule
  185. - `--ppo-epochs`: PPO update epochs per batch
  186. - `--action-top-k`: number of skills selected per step
  187. - `--reward-metric`: `f1` or `llm_judge`
  188. - `--llm-judge-model`: model used for LLM-judge scoring during training and text-dataset evaluation
  189. **Designer / evolution**
  190. - `--enable-designer`: enable skill evolution
  191. - `--designer-freq`: evolve every N outer epochs
  192. - `--designer-max-changes`: max changes per evolution cycle
  193. - `--designer-new-skill-hint`: encourage adding new skills
  194. **Eval / checkpoints**
  195. - `--eval-only`: run evaluation only
  196. - `--load-checkpoint`: checkpoint path (Training checkpoints are saved at outer-epoch boundaries; resuming with `--load-checkpoint` continues from the next outer epoch instead of restarting from epoch 0)
  197. - `--resume-new-wandb-run`: resume training state from a checkpoint but log to a fresh W&B run instead of the saved run ID.
  198. - `--skip-load-operation-bank` / `--skip-load-snapshot-manager`: partial-load options for special cases; do not use them for full resume training.
  199. - `--save-dir`: where to save checkpoints
  200. - `--out-file`: output result file
  201. - `--overwrite`: force rebuilding memory caches under the run's `memories/` directory instead of reusing existing ones
  202. - `--device`: `cuda` or `cpu`
  203. **ALFWorld‑specific**
  204. - `--alfworld-offline-data`: offline trajectories
  205. - `--alfworld-eval-file`: eval split
  206. - `--alfworld-eval-query-source`: `objective` or `first_observation`
  207. **W&B (logging)**
  208. - `--wandb-project`: W&B project name.
  209. - `--wandb-run-name`: W&B run name.
  210. - `--wandb-key`: W&B API key (optional; you can also set `WANDB_API_KEY` in the environment).
  211. ❗For the full list of configuration options, see `src/config.py`.
  212. ## 🙏 Acknowledgments
  213. We thank the authors and maintainers of **[LoCoMo](https://github.com/snap-research/locomo)**, **[LongMemEval](https://github.com/xiaowu0162/LongMemEval)**, **[HotpotQA-Modified](https://huggingface.co/datasets/BytedTsinghua-SIA/hotpotqa/tree/main) (Source: [HotpotQA](https://hotpotqa.github.io/))**, and **[ALFWorld](https://github.com/alfworld/alfworld)** for releasing their datasets, evaluation protocols, and supporting code. Their efforts in building and open-sourcing high-quality benchmarks make it possible to develop, evaluate, and reproduce research on agent memory. We also appreciate the broader open-source LLM/Agent memory community for tooling that supports reliable evaluation and large-scale experimentation.
  214. ## 📚 Citation
  215. ```bibtex
  216. @article{zhang2026memskill,
  217. title={MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents},
  218. author={Zhang, Haozhen and Long, Quanyu and Bao, Jianzhu and Feng, Tao and Zhang, Weizhi and Yue, Haodong and Wang, Wenya},
  219. journal={arXiv preprint arXiv:2602.02474},
  220. year={2026}
  221. }
  222. ```

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.