Skip to content
Back to search

hipporag

MarketplaceWorks with Claude CodeGitHub

[NeurIPS'24] HippoRAG is a novel RAG framework inspired by human long-term memory that enables LLMs to continuously integrate knowledge across external documents. RAG + Knowledge Graphs + Personalized PageRank.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/osu-nlp-group--hipporag/badge.svg)](/listings/osu-nlp-group--hipporag)

HTML

<a href="/listings/osu-nlp-group--hipporag"><img src="/api/listings/osu-nlp-group--hipporag/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Use Conda or `uv` to create a Python 3.10 environment. A project-local `.venv` is recommended for development.

conda create -n hipporag python=3.10
conda activate hipporag
pip install hipporag

Set only the environment variables required by the models you use:

export CUDA_VISIBLE_DEVICES=0,1,2,3
export HF_HOME=<path to Huggingface home directory>
export OPENAI_API_KEY=<your openai api key>   # if you want to use OpenAI model

conda activate hipporag

For a project-local environment managed by `uv`:

uv venv --python 3.10 .venv
source .venv/bin/activate
uv pip install -e .

Install optional local-model support only when needed, for example `uv pip install -e '.[vllm]'` or `uv pip install -e '.[gritlm]'`.

HippoRAG supports `openai>=3.3.1,<4`: compatible 3.x client updates are accepted, while a new major version requires an explicit compatibility review. To reproduce the minimum supported SDK baseline, apply the optional constraint without changing the library's public dependency range:

uv pip install -r requirements.txt -c constraints/openai-tested.txt

SDK compatibility should be checked in separate clean environments against both the minimum baseline above and the newest allowed release. The latter can be installed and tested with:

uv pip install -r requirements.txt --upgrade-package openai
PYTHONPATH=src python -m unittest tests.test_openai_sdk_compat

Upgrading existing indexes

Version 2.0.0a5 binds persisted vectors and OpenIE state to the endpoint, deployment, model, normalization, and component identity that produced them. Existing indexes without an `index_manifest.json`, or indexes whose identity no longer matches the active configuration, are rejected rather than mixed silently. Re-index into a fresh `save_dir`; copying or fabricating only the manifest is not a safe migration. When injecting a custom embedding model, extraction LLM, or text preprocessor, set `index_identity` to a stable version string so configuration changes cannot reuse incompatible state. The default OpenIE output limits are `openie_ner_max_tokens=2048` and `openie_triple_max_tokens=4096`, sized to leave room for reasoning tokens; they are part of the OpenIE identity, so an index built with the earlier development defaults (512 and 2048) must either be re-indexed or loaded with those values set explicitly.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 15 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. {
  2. "name": "hipporag",
  3. "owner": { "name": "OSU NLP Group" },
  4. "metadata": {
  5. "description": "HippoRAG plugins for Claude Code."
  6. },
  7. "plugins": [
  8. {
  9. "name": "hipporag",
  10. "source": "./plugins/hipporag",
  11. "description": "Graph-based long-term memory (HippoRAG) for Claude Code: index documents into a persistent knowledge graph and retrieve multi-hop evidence.",
  12. "category": "knowledge"
  13. }
  14. ]
  15. }
README.md · README · 626 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <h1 align="center">HippoRAG 2: From RAG to Memory</h1>
  2. <p align="center">
  3. <img src="https://github.com/OSU-NLP-Group/HippoRAG/raw/main/images/hippo_brain.png" width="55%" style="max-width: 300px;">
  4. </p>
  5. [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/drive/1nuelysWsXL8F5xH6q4JYJI8mvtlmeM9O#scrollTo=TjHdNe2KC81K)
  6. [<img align="center" src="https://img.shields.io/badge/arXiv-2502.14802 HippoRAG 2-b31b1b" />](https://arxiv.org/abs/2502.14802)
  7. [<img align="center" src="https://img.shields.io/badge/🤗 Dataset-HippoRAG 2-yellow" />](https://huggingface.co/datasets/osunlp/HippoRAG_2/tree/main)
  8. [<img align="center" src="https://img.shields.io/badge/arXiv-2405.14831 HippoRAG 1-b31b1b" />](https://arxiv.org/abs/2405.14831)
  9. [<img align="center" src="https://img.shields.io/badge/GitHub-HippoRAG 1-blue" />](https://github.com/OSU-NLP-Group/HippoRAG/tree/legacy)
  10. HippoRAG 2 is a memory framework for LLMs that recognizes and uses connections in new knowledge, mirroring a key function of human long-term memory.
  11. Our experiments show that HippoRAG 2 improves associativity (multi-hop retrieval) and sense-making (the process of integrating large and complex contexts) in even the most advanced RAG systems, without sacrificing their performance on simpler tasks.
  12. Like its predecessor, HippoRAG 2 remains cost and latency efficient in online processes, while using significantly fewer resources for offline indexing compared to other graph-based solutions such as GraphRAG, RAPTOR, and LightRAG.
  13. <p align="center">
  14. <img align="center" src="https://github.com/OSU-NLP-Group/HippoRAG/raw/main/images/intro.png" />
  15. </p>
  16. <p align="center">
  17. <b>Figure 1:</b> Evaluation of continual learning capabilities across three key dimensions: factual memory (NaturalQuestions, PopQA), sense-making (NarrativeQA), and associativity (MuSiQue, 2Wiki, HotpotQA, and LV-Eval). HippoRAG 2 surpasses other methods across all
  18. categories, bringing it one step closer to true long-term memory.
  19. </p>
  20. <p align="center">
  21. <img align="center" src="https://github.com/OSU-NLP-Group/HippoRAG/raw/main/images/methodology.png" />
  22. </p>
  23. <p align="center">
  24. <b>Figure 2:</b> HippoRAG 2 methodology.
  25. </p>
  26. ### Papers
  27. * [**HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models**](https://arxiv.org/abs/2405.14831) [NeurIPS '24].
  28. * [**From RAG to Memory: Non-Parametric Continual Learning for Large Language Models**](https://arxiv.org/abs/2502.14802) [ICML '25].
  29. ----
  30. ## Installation
  31. Use Conda or `uv` to create a Python 3.10 environment. A project-local `.venv` is recommended for development.
  32. ```sh
  33. conda create -n hipporag python=3.10
  34. conda activate hipporag
  35. pip install hipporag
  36. ```
  37. Set only the environment variables required by the models you use:
  38. ```sh
  39. export CUDA_VISIBLE_DEVICES=0,1,2,3
  40. export HF_HOME=<path to Huggingface home directory>
  41. export OPENAI_API_KEY=<your openai api key> # if you want to use OpenAI model
  42. conda activate hipporag
  43. ```
  44. For a project-local environment managed by `uv`:
  45. ```sh
  46. uv venv --python 3.10 .venv
  47. source .venv/bin/activate
  48. uv pip install -e .
  49. ```
  50. Install optional local-model support only when needed, for example `uv pip install -e '.[vllm]'` or `uv pip install -e '.[gritlm]'`.
  51. HippoRAG supports `openai>=3.3.1,<4`: compatible 3.x client updates are accepted,
  52. while a new major version requires an explicit compatibility review. To reproduce
  53. the minimum supported SDK baseline, apply the optional constraint without changing
  54. the library's public dependency range:
  55. ```sh
  56. uv pip install -r requirements.txt -c constraints/openai-tested.txt
  57. ```
  58. SDK compatibility should be checked in separate clean environments against both
  59. the minimum baseline above and the newest allowed release. The latter can be
  60. installed and tested with:
  61. ```sh
  62. uv pip install -r requirements.txt --upgrade-package openai
  63. PYTHONPATH=src python -m unittest tests.test_openai_sdk_compat
  64. ```
  65. ### Upgrading existing indexes
  66. Version 2.0.0a5 binds persisted vectors and OpenIE state to the endpoint, deployment, model, normalization, and component identity that produced them. Existing indexes without an `index_manifest.json`, or indexes whose identity no longer matches the active configuration, are rejected rather than mixed silently. Re-index into a fresh `save_dir`; copying or fabricating only the manifest is not a safe migration. When injecting a custom embedding model, extraction LLM, or text preprocessor, set `index_identity` to a stable version string so configuration changes cannot reuse incompatible state. The default OpenIE output limits are `openie_ner_max_tokens=2048` and `openie_triple_max_tokens=4096`, sized to leave room for reasoning tokens; they are part of the OpenIE identity, so an index built with the earlier development defaults (512 and 2048) must either be re-indexed or loaded with those values set explicitly.
  67. ## Quick Start
  68. ### OpenAI
  69. The complete runnable version is [`examples/demo_openai.py`](examples/demo_openai.py). A minimal workflow is:
  70. Set `OPENAI_API_KEY` before using OpenAI models:
  71. ```sh
  72. export OPENAI_API_KEY=<your OpenAI API key>
  73. ```
  74. ```python
  75. from hipporag import HippoRAG
  76. docs = ["George Rankin is a politician."]
  77. queries = ["What is George Rankin's occupation?"]
  78. with HippoRAG(save_dir="outputs", llm_model_name="gpt-4o-mini", embedding_model_name="text-embedding-3-small") as hipporag:
  79. hipporag.index(docs=docs)
  80. results = hipporag.rag_qa(queries=queries)
  81. ```
  82. #### OpenAI-compatible endpoints
  83. Pass custom base URLs for OpenAI-compatible LLM and embedding servers, such as a local vLLM server or a hosted gateway. No registration is needed for any OpenAI-compatible endpoint:
  84. ```python
  85. hipporag = HippoRAG(
  86. save_dir=save_dir,
  87. llm_model_name="your-llm",
  88. llm_base_url="http://localhost:8000/v1",
  89. llm_api_key_env="MY_LLM_API_KEY", # optional; defaults to OPENAI_API_KEY
  90. embedding_model_name="your-embedding-model",
  91. embedding_provider="openai",
  92. embedding_base_url="http://localhost:8001/v1",
  93. embedding_api_key_env="MY_EMBEDDING_API_KEY", # optional; defaults to OPENAI_API_KEY
  94. )
  95. ```
  96. API keys are read from `OPENAI_API_KEY` by default. Set `llm_api_key_env` or `embedding_api_key_env` to the name of another environment variable when the LLM and embedding endpoints need different keys, for example a gateway LLM with OpenAI embeddings. When the relevant variable is unset and the endpoint is a loopback address that does not require authentication, HippoRAG supplies a client-local placeholder without changing the process-wide `OPENAI_API_KEY` environment variable.
  97. Official OpenAI and Azure endpoints receive `max_completion_tokens`, the gateways listed under [Gateway prefixes](#gateway-prefixes) receive the parameter they document (also when reached through `llm_base_url`), and other endpoints receive the more widely supported `max_tokens`. Set `llm_supports_max_completion_tokens=True` in `BaseConfig` if your endpoint requires `max_completion_tokens`, for example for some reasoning models.
  98. OpenAI-compatible chat and embedding endpoints must return standard `usage` data. HippoRAG fails closed instead of caching a response whose token cost cannot be accounted for.
  99. To send a single test request through an endpoint:
  100. ```sh
  101. python examples/demo_gateway.py --llm_name <vendor/model> --llm_base_url <base URL> --llm_api_key_env MY_LLM_API_KEY
  102. ```
  103. #### Gateway prefixes
  104. A few gateways also have a model-name prefix that fills in their endpoint, key variable and token parameter. The prefix is stripped before the request is sent, and the request is identical to the generic configuration above; `llm_base_url` and `llm_api_key_env` still override the defaults.
  105. | Prefix | Equivalent `llm_base_url` | Default key variable |
  106. | --- | --- | --- |
  107. | `orcarouter/` | `https://api.orcarouter.ai/v1` | `ORCAROUTER_API_KEY` |
  108. | `atlascloud/` | `https://api.atlascloud.ai/v1` | `ATLASCLOUD_API_KEY` |
  109. These prefixes are kept for compatibility and are listed in the order they were added; they are not an endorsement, and HippoRAG is not affiliated with any gateway. New OpenAI-compatible gateways should use the generic configuration rather than a new prefix (see [CONTRIBUTING.md](CONTRIBUTING.md)).
  110. ### Amazon Bedrock
  111. Models available through the standard Bedrock Runtime endpoint use the existing LiteLLM route. Prefix the Bedrock model ID with `bedrock/`, as shown in `examples/demo_bedrock.py`.
  112. OpenAI models such as GPT-5.5 use Amazon Bedrock Mantle and its Responses API instead. Create a Bedrock API key, then run:
  113. ```sh
  114. export AWS_BEARER_TOKEN_BEDROCK=<your Bedrock API key>
  115. python examples/demo_bedrock_mantle.py
  116. ```
  117. The corresponding configuration is:
  118. ```python
  119. hipporag = HippoRAG(
  120. save_dir='outputs/bedrock-mantle',
  121. llm_model_name='bedrock-mantle/openai.gpt-5.5',
  122. llm_base_url='https://bedrock-mantle.us-east-2.api.aws/openai/v1',
  123. embedding_model_name=embedding_model_name,
  124. )
  125. ```
  126. The Mantle endpoint and model availability are region-specific. HippoRAG requires an explicit endpoint and raises an error if the Bedrock API key is missing. To use an existing AWS profile instead, construct a `BaseConfig` with `bedrock_mantle_auth='aws_credentials'`, `bedrock_aws_profile='<profile>'`, and `bedrock_region='<region>'`; this explicitly enables SigV4 authentication. Mantle response storage is disabled by default (`store=False`); pass `store=True` to `infer` only when server-side conversation state is required.
  127. ### Local Deployment (vLLM)
  128. This simple example will illustrate how to use `hipporag` with any vLLM-compatible locally deployed LLM.
  129. 1. Run a local [OpenAI-compatible vLLM server](https://docs.vllm.ai/en/latest/getting_started/quickstart.html#quickstart-online) with specified GPUs (make sure you leave enough memory for your embedding model).
  130. Keep `VLLM_WORKER_MULTIPROC_METHOD=spawn` when running vLLM with multiple GPUs; it makes the required multiprocessing mode explicit and avoids CUDA initialization problems with forked workers.
  131. ```sh
  132. export CUDA_VISIBLE_DEVICES=0,1
  133. export VLLM_WORKER_MULTIPROC_METHOD=spawn
  134. export HF_HOME=<path to Huggingface home directory>
  135. conda activate hipporag # vllm should be in this environment
  136. # Tune gpu-memory-utilization or max_model_len to fit your GPU memory, if OOM occurs
  137. vllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 2 --max_model_len 4096 --gpu-memory-utilization 0.95
  138. ```
  139. 2. Point `hipporag` at the server. The model name must match the one passed to `vllm serve`; see [OpenAI-compatible endpoints](#openai-compatible-endpoints) for API keys and other options, and [`examples/demo_local.py`](examples/demo_local.py) for a runnable version:
  140. ```python
  141. from hipporag import HippoRAG
  142. hipporag = HippoRAG(
  143. save_dir="outputs", # each LLM/embedding model combination gets its own subdirectory
  144. llm_model_name="meta-llama/Llama-3.3-70B-Instruct",
  145. llm_base_url="http://localhost:8000/v1",
  146. embedding_model_name="nvidia/NV-Embed-v2", # or another supported embedding model, e.g. GritLM/GritLM-7B
  147. )
  148. # Same indexing, retrieval and QA as with OpenAI models above
  149. ```
  150. ## MCP Server
  151. HippoRAG can serve a persistent index to agents and MCP clients such as Claude Code, Claude Desktop, Cursor or VS Code through the [Model Context Protocol](https://modelcontextprotocol.io). Install the optional extra:
  152. ```sh
  153. pip install 'hipporag[mcp]'
  154. ```
  155. The server exposes these tools:
  156. | Tool | Description |
  157. | --- | --- |
  158. | `retrieve` | Return the `top_k` most relevant passages for a query, with scores and source metadata. |
  159. | `rag_qa` | Answer a question with the configured LLM, grounded in retrieved passages. |
  160. | `index` | Add documents, as plain strings or `{"text", "source_id", "metadata"}` objects. Passages already in the index are skipped. |
  161. | `delete` | Remove passages by exact text or by `source_id`. |
  162. | `index_stats` | Report passage, entity, fact and graph counts. |
  163. The active configuration, with secrets redacted, is available as the resource `hipporag://config`. The index is stored in `--save_dir` and persists across restarts. The same LLM and embedding settings must be used every time the server opens an existing index (see [Upgrading existing indexes](#upgrading-existing-indexes)).
  164. For a local client using stdio, for example Claude Code:
  165. ```sh
  166. claude mcp add hipporag -e OPENAI_API_KEY=<your OpenAI API key> -- hipporag-mcp --save_dir /path/to/index --llm_name gpt-4o-mini --embedding_model_name text-embedding-3-small
  167. ```
  168. Other clients use the equivalent JSON configuration:
  169. ```json
  170. {
  171. "mcpServers": {
  172. "hipporag": {
  173. "command": "hipporag-mcp",
  174. "args": ["--save_dir", "/path/to/index", "--llm_name", "gpt-4o-mini", "--embedding_model_name", "text-embedding-3-small"],
  175. "env": {"OPENAI_API_KEY": "<your OpenAI API key>"}
  176. }
  177. }
  178. }
  179. ```
  180. To serve over Streamable HTTP at `http://127.0.0.1:8000/mcp`, add `--transport streamable-http` (with `--host` and `--port` if needed). The HTTP server has no authentication. Keep it on a loopback address, or put it behind an authenticating proxy and add `--read_only` to disable the `index` and `delete` tools.
  181. LLM and embedding endpoints are configured with the same options as the Python API: `--llm_base_url`, `--llm_api_key_env`, `--embedding_base_url`, `--embedding_api_key_env`, `--embedding_provider`, `--azure_endpoint` and `--azure_embedding_endpoint`. Any other `BaseConfig` field can be set with a repeatable `--config KEY=VALUE`, for example `--config retrieval_top_k=100 --config vector_store_type=qdrant`. Run `hipporag-mcp --help` for the full list.
  182. Tool calls are executed one at a time on a dedicated worker thread, so a long `index` call delays later calls but does not block the MCP session.
  183. ### Claude Code Plugin
  184. This repository is also a Claude Code plugin marketplace. The plugin bundles the MCP server with a skill that teaches Claude when to use each tool, plus a `/hipporag:index-folder` command. Install it from a Claude Code session:
  185. ```
  186. /plugin marketplace add OSU-NLP-Group/HippoRAG
  187. /plugin install hipporag@hipporag
  188. ```
  189. Claude Code prompts for the API key (stored in the system keychain), the LLM and embedding model names, and optional OpenAI-compatible base URLs. The server is launched with [`uvx`](https://docs.astral.sh/uv/), so [uv](https://docs.astral.sh/uv/getting-started/installation/) must be installed; HippoRAG itself does not need to be. The index is stored in the plugin's data directory (`~/.claude/plugins/data/`) and is shared across projects.
  190. The first launch installs HippoRAG and its dependencies, including PyTorch, which can take longer than Claude Code's MCP startup timeout. If the server fails to connect the first time, pre-install it once and restart Claude Code:
  191. ```sh
  192. uvx --python 3.12 --from 'hipporag[mcp] @ git+https://github.com/OSU-NLP-Group/HippoRAG' hipporag-mcp --help
  193. ```
  194. Changing the LLM or embedding model switches to a separate index, because each index is tied to the models it was built with.
  195. ## Vector Store Backends
  196. HippoRAG stores embeddings in local Parquet files by default. It can also use
  197. Qdrant, ChromaDB, or Milvus through `BaseConfig.vector_store_type`.
  198. Remote collections are isolated by a per-index namespace and the backend endpoint
  199. is fingerprinted in `index_manifest.json`. Set a stable, unique
  200. `vector_store_namespace` when an index must keep the same remote collection after
  201. its local `save_dir` is moved; do not reuse a namespace for unrelated indexes.
  202. ### Milvus
  203. Install the optional Milvus dependency when you want to use this backend:
  204. ```sh
  205. pip install "hipporag[milvus]"
  206. ```
  207. For a source checkout, install the optional extra from the repository root:
  208. ```sh
  209. pip install -e ".[milvus]"
  210. ```
  211. Milvus Lite is used by default and stores its local database inside the
  212. HippoRAG working directory, so no separate server is required:
  213. ```python
  214. from hipporag import HippoRAG
  215. from hipporag.utils.config_utils import BaseConfig
  216. config = BaseConfig(vector_store_type="milvus")
  217. hipporag = HippoRAG(
  218. global_config=config,
  219. save_dir=save_dir,
  220. llm_model_name=llm_model_name,
  221. embedding_model_name=embedding_model_name,
  222. )
  223. ```
  224. To connect to Milvus server or Zilliz Cloud, set `milvus_uri` and
  225. `milvus_token` directly or use `MILVUS_URI` and `MILVUS_TOKEN` environment
  226. variables. `MILVUS_DB_NAME` and `MILVUS_CONSISTENCY_LEVEL` are also honored
  227. when set:
  228. ```python
  229. config = BaseConfig(
  230. vector_store_type="milvus",
  231. milvus_uri="http://localhost:19530",
  232. milvus_token=None,
  233. milvus_db_name=None,
  234. milvus_consistency_level="Session",
  235. )
  236. ```
  237. ## Testing
  238. Run the offline unit tests before submitting changes:
  239. ```sh
  240. python -m unittest discover -s tests -p 'test_*.py'
  241. python tests/integration/run_vector_stores.py
  242. ```
  243. The MCP server tests in `tests/test_mcp_server.py` are skipped unless the `mcp` extra is installed. They start the real `hipporag-mcp` command over stdio and Streamable HTTP against a local fake OpenAI-compatible endpoint, so they need no API key.
  244. Provider integration scripts exercise indexing, graph reload, incremental updates, and deletion. They require the corresponding API or local model service:
  245. | Provider | Command |
  246. | --- | --- |
  247. | OpenAI | `python tests/integration/run_openai.py` |
  248. | Azure OpenAI | `python tests/integration/run_azure.py --azure_endpoint <resource-url> --azure_api_version <version> --azure_embedding_endpoint <resource-url>` |
  249. | OpenAI-compatible endpoints | `python tests/integration/run_gateway.py --llm_model_name <vendor/model> --llm_base_url <base URL> --llm_api_key_env <key variable>` |
  250. | Local vLLM | `python tests/integration/run_local.py` |
  251. | Transformers | `python tests/integration/run_transformers.py` |
  252. The local integration script expects an OpenAI-compatible server at `http://localhost:6578/v1`. See [Local Deployment](#local-deployment-vllm) for server setup.
  253. ## Reproducing our Experiments
  254. To use our code to run experiments we recommend you clone this repository and follow the structure of the `main.py` script.
  255. ### Data for Reproducibility
  256. We evaluated several sampled datasets in our paper, some of which are already included in the `reproduce/dataset` directory of this repo. For the complete set of datasets, please visit
  257. our [HuggingFace dataset](https://huggingface.co/datasets/osunlp/HippoRAG_v2) and place them under `reproduce/dataset`. We also provide the OpenIE results for both `gpt-4o-mini` and `Llama-3.3-70B-Instruct` for our `musique` sample under `outputs/musique`.
  258. To test your environment is properly set up, you can use the small dataset `reproduce/dataset/sample.json` for debugging as shown below.
  259. ### Running indexing and QA
  260. Initialize the environmental variables and activate the environment:
  261. ```sh
  262. export CUDA_VISIBLE_DEVICES=0,1,2,3
  263. export HF_HOME=<path to Huggingface home directory>
  264. export OPENAI_API_KEY=<your openai api key> # if you want to use OpenAI model
  265. conda activate hipporag
  266. ```
  267. ### OpenAI
  268. ```sh
  269. dataset=sample # or any other dataset under `reproduce/dataset`
  270. # Run OpenAI model
  271. python main.py --dataset $dataset --llm_base_url https://api.openai.com/v1 --llm_name gpt-4o-mini --embedding_name nvidia/NV-Embed-v2
  272. ```
  273. Azure OpenAI uses the same entry point:
  274. ```sh
  275. export AZURE_OPENAI_API_KEY=<your Azure OpenAI API key>
  276. python main.py --dataset sample --llm_name gpt-4o-mini --embedding_name text-embedding-3-small \
  277. --azure_endpoint https://<resource>.openai.azure.com --azure_api_version <api-version> \
  278. --azure_chat_deployment <chat-deployment> \
  279. --azure_embedding_endpoint https://<resource>.openai.azure.com \
  280. --azure_embedding_deployment <embedding-deployment>
  281. ```
  282. If embeddings use a different API version, also set `--azure_embedding_api_version`. Legacy full operation URLs containing the deployment and `api-version` remain accepted temporarily, but emit a migration warning.
  283. To run the standard dense-retrieval/DPR-style baseline, add `--rag_type standard`. Both methods share the same loading and evaluation logic:
  284. ```sh
  285. python main.py --dataset sample --rag_type standard --embedding_batch_size 1
  286. ```
  287. ### Run with vLLM (Llama)
  288. 1. As above, run a local [OpenAI-compatible vLLM server](https://docs.vllm.ai/en/latest/getting_started/quickstart.html#quickstart-online) with specified GPU.
  289. Do not omit `VLLM_WORKER_MULTIPROC_METHOD=spawn` when using multiple GPUs.
  290. ```sh
  291. export CUDA_VISIBLE_DEVICES=0,1
  292. export VLLM_WORKER_MULTIPROC_METHOD=spawn
  293. export HF_HOME=<path to Huggingface home directory>
  294. conda activate hipporag # vllm should be in this environment
  295. # Tune gpu-memory-utilization or max_model_len to fit your GPU memory, if OOM occurs
  296. vllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 2 --max_model_len 4096 --gpu-memory-utilization 0.95
  297. ```
  298. 2. Use another GPUs to run the main program in another terminal.
  299. ```sh
  300. export CUDA_VISIBLE_DEVICES=2,3 # set another GPUs while vLLM server is running
  301. export HF_HOME=<path to Huggingface home directory>
  302. dataset=sample
  303. python main.py --dataset $dataset --llm_base_url http://localhost:8000/v1 --llm_name meta-llama/Llama-3.3-70B-Instruct --embedding_name nvidia/NV-Embed-v2
  304. ```
  305. #### Advanced: vLLM offline batch
  306. vLLM offers an [offline batch mode](https://docs.vllm.ai/en/latest/getting_started/quickstart.html#offline-batched-inference) for faster inference, which could bring us more than 3x faster indexing compared to vLLM online server.
  307. The offline implementation also sets `VLLM_WORKER_MULTIPROC_METHOD=spawn` internally, and the export below documents the required multiprocessing mode explicitly.
  308. 1. Use the following command to run the main program with vLLM offline batch mode.
  309. ```sh
  310. export CUDA_VISIBLE_DEVICES=0,1,2,3 # use all GPUs for faster offline indexing
  311. export VLLM_WORKER_MULTIPROC_METHOD=spawn
  312. export HF_HOME=<path to Huggingface home directory>
  313. export OPENAI_API_KEY=''
  314. dataset=sample
  315. python main.py --dataset $dataset --llm_name meta-llama/Llama-3.3-70B-Instruct --openie_mode offline
  316. ```
  317. 2. After the first step, OpenIE result is saved to file. Go back to run vLLM online server and main program as described in the `Run with vLLM (Llama)` main section.
  318. ## Debugging Note
  319. - `/reproduce/dataset/sample.json` is a small dataset specifically for debugging.
  320. - When debugging vLLM offline mode, set `tensor_parallel_size` as `1` in `hipporag/llm/vllm_offline.py`.
  321. - If you want to rerun a particular experiment, remember to clear the saved files, including OpenIE results and knowledge graph, e.g.,
  322. ```sh
  323. rm reproduce/dataset/openie_results/openie_sample_results_ner_meta-llama_Llama-3.3-70B-Instruct_3.json
  324. rm -rf outputs/sample/sample_meta-llama_Llama-3.3-70B-Instruct_nvidia_NV-Embed-v2
  325. ```
  326. ### Custom Datasets
  327. To setup your own custom dataset for evaluation, follow the format and naming convention shown in `reproduce/dataset/sample_corpus.json` (your dataset's name should be followed by `_corpus.json`). If running an experiment with pre-defined questions, organize your query corpus according to the query file `reproduce/dataset/sample.json`, be sure to also follow our naming convention.
  328. The corpus and optional query JSON files should have the following format:
  329. #### Retrieval Corpus JSON
  330. ```json
  331. [
  332. {
  333. "title": "FIRST PASSAGE TITLE",
  334. "text": "FIRST PASSAGE TEXT",
  335. "idx": 0
  336. },
  337. {
  338. "title": "SECOND PASSAGE TITLE",
  339. "text": "SECOND PASSAGE TEXT",
  340. "idx": 1
  341. }
  342. ]
  343. ```
  344. #### (Optional) Query JSON
  345. ```json
  346. [
  347. {
  348. "id": "sample/question_1.json",
  349. "question": "QUESTION",
  350. "answer": [
  351. "ANSWER"
  352. ],
  353. "answerable": true,
  354. "paragraphs": [
  355. {
  356. "title": "{FIRST SUPPORTING PASSAGE TITLE}",
  357. "text": "{FIRST SUPPORTING PASSAGE TEXT}",
  358. "is_supporting": true,
  359. "idx": 0
  360. },
  361. {
  362. "title": "{SECOND SUPPORTING PASSAGE TITLE}",
  363. "text": "{SECOND SUPPORTING PASSAGE TEXT}",
  364. "is_supporting": true,
  365. "idx": 1
  366. }
  367. ]
  368. }
  369. ]
  370. ```
  371. #### (Optional) Chunking Corpus
  372. When preparing your data, you may need to chunk each passage, as longer passage may be too complex for the OpenIE process.
  373. ## Code Structure
  374. ```
  375. 📦 .
  376. │-- 📂 src/hipporag
  377. │ ├── 📂 embedding_model # Implementation of all embedding models
  378. │ │ ├── __init__.py # Getter function for get specific embedding model classes
  379. | | ├── base.py # Base embedding model class `BaseEmbeddingModel` to inherit and `EmbeddingConfig`
  380. | | ├── NVEmbedV2.py # Implementation of NV-Embed-v2 model
  381. | | ├── ...
  382. │ ├── 📂 evaluation # Implementation of all evaluation metrics
  383. │ │ ├── __init__.py
  384. | | ├── base.py # Base evaluation metric class `BaseMetric` to inherit
  385. │ │ ├── qa_eval.py # Eval metrics for QA
  386. │ │ ├── retrieval_eval.py # Eval metrics for retrieval
  387. │ ├── 📂 information_extraction # Implementation of all information extraction models
  388. │ │ ├── __init__.py
  389. | | ├── openie_openai.py # Online OpenIE through any chat LLM class
  390. | | ├── openie_vllm_offline.py # Model for OpenIE with LLMs deployed offline with vLLM
  391. | | ├── openie_transformers_offline.py # Offline OpenIE with Transformers
  392. │ ├── 📂 llm # Classes for inference with large language models
  393. │ │ ├── __init__.py # Getter function
  394. | | ├── base.py # Config class for LLM inference and base LLM inference class to inherit
  395. | | ├── openai_gpt.py # OpenAI, Azure OpenAI and any OpenAI-compatible endpoint (e.g. a vLLM server)
  396. | | ├── gateways.py # Frozen registry of gateway prefixes and the shared max-token parameter rule
  397. | | ├── gateway_llm.py # Class for the registered gateway prefixes
  398. | | ├── bedrock_llm.py # Amazon Bedrock Runtime through LiteLLM
  399. | | ├── bedrock_mantle.py # Amazon Bedrock Mantle (Responses API)
  400. | | ├── transformers_llm.py # Local inference with Transformers
  401. | | ├── vllm_offline.py # Class for inference using the vLLM API directly
  402. │ ├── 📂 prompts # Prompt templates and prompt template manager class
  403. | │ ├── 📂 dspy_prompts # Prompts for filtering
  404. | │ │ ├── ...
  405. | │ ├── 📂 templates # All prompt templates for template manager to load
  406. | │ │ ├── README.md # Documentations of usage of prompte template manager and prompt template files
  407. | │ │ ├── __init__.py
  408. | │ │ ├── triple_extraction.py
  409. | │ │ ├── ...
  410. │ │ ├── __init__.py
  411. | | ├── linking.py # Instruction for linking
  412. | | ├── prompt_template_manager.py # Implementation of prompt template manager
  413. │ ├── 📂 utils # All utility functions used across this repo (the file name indicates its relevant usage)
  414. │ │ ├── config_utils.py # We use only one config across all modules and its setup is specified here
  415. | | ├── ...
  416. │ ├── __init__.py
  417. │ ├── HippoRAG.py # Highest level class for initiating retrieval, question answering, and evaluations
  418. │ ├── mcp_server.py # MCP server (`hipporag-mcp`) exposing an index to agents
  419. │ ├── embedding_store.py # Storage database to load, manage and save embeddings for passages, entities and facts.
  420. │ ├── rerank.py # Reranking and filtering methods
  421. │-- 📂 examples # Minimal provider-specific usage examples
  422. │-- 📂 tests
  423. │ ├── 📂 integration # Manual provider and vector-store integration checks
  424. │ ├── test_gateway_llm.py
  425. │ ├── test_regressions.py
  426. │ ├── ...
  427. │-- 📂 reproduce/dataset # Sample and paper evaluation datasets
  428. │-- 📂 plugins/hipporag # Claude Code plugin (MCP server config and skills)
  429. │-- 📂 .claude-plugin # Claude Code plugin marketplace manifest
  430. │-- 📜 main.py # Unified HippoRAG, Azure, and standard-RAG experiment entry point
  431. │-- 📜 README.md
  432. │-- 📜 requirements.txt # Dependencies list
  433. │-- 📜 .gitignore # Files to exclude from Git
  434. ```
  435. ## Contact
  436. Questions or issues? File an issue or contact
  437. [Bernal Jiménez Gutiérrez](mailto:jimenezgutierrez.1@osu.edu),
  438. [Yiheng Shu](mailto:shu.251@osu.edu),
  439. [Yu Su](mailto:su.809@osu.edu),
  440. The Ohio State University
  441. ## Citation
  442. If you find this work useful, please consider citing our papers:
  443. ### HippoRAG 2
  444. ```
  445. @misc{gutiérrez2025ragmemorynonparametriccontinual,
  446. title={From RAG to Memory: Non-Parametric Continual Learning for Large Language Models},
  447. author={Bernal Jiménez Gutiérrez and Yiheng Shu and Weijian Qi and Sizhe Zhou and Yu Su},
  448. year={2025},
  449. eprint={2502.14802},
  450. archivePrefix={arXiv},
  451. primaryClass={cs.CL},
  452. url={https://arxiv.org/abs/2502.14802},
  453. }
  454. ```
  455. ### HippoRAG
  456. ```
  457. @inproceedings{gutiérrez2024hipporag,
  458. title={HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models},
  459. author={Bernal Jiménez Gutiérrez and Yiheng Shu and Yu Gu and Michihiro Yasunaga and Yu Su},
  460. booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
  461. year={2024},
  462. url={https://openreview.net/forum?id=hkujvAPVsg}
  463. ```
  464. ## TODO:
  465. - [x] Add support for more embedding models
  466. - [x] Add support for embedding endpoints
  467. - [x] Add support for vector database integration
  468. Please feel free to open an issue or PR if you have any questions or suggestions.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.