Skip to content
Back to search

langroid

MarketplaceWorks with Claude CodeGitHub

Harness LLMs with Multi-Agent Programming

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/langroid/langroid with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/langroid/langroid'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/langroid--langroid/badge.svg)](/listings/langroid--langroid)

HTML

<a href="/listings/langroid--langroid"><img src="/api/listings/langroid--langroid/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Install `langroid`

Langroid requires Python 3.11+. We recommend using a virtual environment. Use `pip` to install a bare-bones slim version of `langroid` (from PyPi) to your virtual environment:

pip install langroid

The core Langroid package lets you use OpenAI Embeddings models via their API. If you instead want to use the `sentence-transformers` embedding models from HuggingFace, install Langroid like this:

pip install "langroid[hf-embeddings]"

For many practical scenarios, you may need additional optional dependencies: - To use various document-parsers, install langroid with the `doc-chat` extra: ```bash pip install "langroid[doc-chat]" ``` - For "chat with databases", use the `db` extra: ```bash pip install "langroid[db]" ``` - You can specify multiple extras by separating them with commas, e.g.: ```bash pip install "langroid[doc-chat,db]" ``` - To simply install _all_ optional dependencies, use the `all` extra (but note that this will result in longer load/startup times and a larger install size): ```bash pip install "langroid[all]" ``` <details> <summary><b>Optional Installs for using SQL Chat with a PostgreSQL DB </b></summary>

If you are using `SQLChatAgent` (e.g. the script [`examples/data-qa/sql-chat/sql_chat.py`](https://github.com/langroid/langroid/blob/main/examples/data-qa/sql-chat/sql_chat.py)), with a postgres db, you will need to:

- Install PostgreSQL dev libraries for your platform, e.g. - `sudo apt-get install libpq-dev` on Ubuntu, - `brew install postgresql` on Mac, etc. - Install langroid with the postgres extra, e.g. `pip install langroid[postgres]` or `poetry add "langroid[postgres]"` or `poetry install -E postgres`, (or the corresponding `uv` versions, e.g. `uv add "langroid[postgres]"` or `uv pip install langroid[postgres]`). If this gives you an error, try `pip install psycopg2-binary` in your virtualenv. </details>

๐Ÿ“ If you get strange errors involving `mysqlclient`, try doing `pip uninstall mysqlclient` followed by `pip install mysqlclient`.

Claude Code Plugin (Optional)

This plugin provides two skills:

- `langroid:patterns` - Your Claude Code agent can leverage this skill to produce Langroid multi-agent code using proper design patterns. - `langroid:add-pattern` - The agent can use this skill to record new patterns it learns, for future reference, either autonomously or when prompted by the user.

**Step 1: Add the Langroid marketplace**

From terminal:

claude plugin marketplace add langroid/langroid

Or within Claude Code:

/plugin marketplace add langroid/langroid

**Step 2: Install the Langroid plugin**

From terminal:

claude plugin install langroid@langroid

Or within Claude Code:

/plugin install langroid@langroid

Once installed, simply ask your Claude Code agent to implement Langroid patterns in natural language, e.g.,

> set up a Langroid agent so it uses the EditTool, and wrap it in a task that ends as soon as the tool is generated

and it will automatically use the `langroid:patterns` skill to follow the right design pattern.

You can also ask Claude Code to record a new pattern when you discover one, e.g.,

> record this as a new Langroid pattern for setting up MCP tools

Set up environment variables (API keys, etc)

To get started, all you need is an OpenAI API Key. If you don't have one, see [this OpenAI Page](https://platform.openai.com/docs/quickstart). (Note that while this is the simplest way to get started, Langroid works with practically any LLM, not just those from OpenAI. See the guides to using [Open/Local LLMs](https://langroid.github.io/langroid/tutorials/local-llm-setup/), and other [non-OpenAI](https://langroid.github.io/langroid/tutorials/non-openai-llms/) proprietary LLMs.)

In the root of the repo, copy the `.env-template` file to a new file `.env`:

cp .env-template .env

Then insert your OpenAI API Key.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json ยท Manifest ยท 14 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. {
  2. "name": "langroid",
  3. "version": "1.0.0",
  4. "owner": {
  5. "name": "Langroid"
  6. },
  7. "plugins": [
  8. {
  9. "name": "langroid",
  10. "source": "./plugins/langroid",
  11. "description": "Langroid multi-agent framework patterns and best practices"
  12. }
  13. ]
  14. }
README.md ยท README ยท 1,111 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. <img src="https://raw.githubusercontent.com/langroid/langroid/main/docs/assets/langroid-card-lambda-ossem-rust-1200-630.png" alt="Logo"
  3. width="400" align="center">
  4. </div>
  5. <div align="center">
  6. [![PyPI - Version](https://img.shields.io/pypi/v/langroid)](https://pypi.org/project/langroid/)
  7. [![Downloads](https://img.shields.io/pypi/dm/langroid)](https://pypi.org/project/langroid/)
  8. [![Pytest](https://github.com/langroid/langroid/actions/workflows/pytest.yml/badge.svg)](https://github.com/langroid/langroid/actions/workflows/pytest.yml)
  9. [![codecov](https://codecov.io/gh/langroid/langroid/graph/badge.svg)](https://codecov.io/gh/langroid/langroid)
  10. [![Multi-Architecture DockerHub](https://github.com/langroid/langroid/actions/workflows/docker-publish.yml/badge.svg)](https://github.com/langroid/langroid/actions/workflows/docker-publish.yml)
  11. [![Static Badge](https://img.shields.io/badge/Documentation-blue?link=https%3A%2F%2Flangroid.github.io%2Flangroid%2F&link=https%3A%2F%2Flangroid.github.io%2Flangroid%2F)](https://langroid.github.io/langroid)
  12. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb)
  13. [![Discord](https://img.shields.io/badge/Discord-%235865F2.svg?style=flat&logo=discord&logoColor=white)](https://discord.gg/ZU36McDgDs)
  14. [![Substack](https://img.shields.io/badge/Substack-%23006f5c.svg?style=flat&logo=substack&logoColor=FF6719)](https://langroid.substack.com/p/langroid-harness-llms-with-multi-agent-programming)
  15. </div>
  16. <h3 align="center">
  17. <a target="_blank"
  18. href="https://langroid.github.io/langroid/" rel="dofollow">
  19. <strong>Documentation</strong></a>
  20. &middot;
  21. <a target="_blank" href="https://github.com/langroid/langroid-examples" rel="dofollow">
  22. <strong>Examples Repo</strong></a>
  23. &middot;
  24. <a target="_blank" href="https://discord.gg/ZU36McDgDs" rel="dofollow">
  25. <strong>Discord</strong></a>
  26. &middot;
  27. <a target="_blank" href="https://github.com/langroid/langroid/blob/main/CONTRIBUTING.md" rel="dofollow">
  28. <strong>Contributing</strong></a>
  29. <br />
  30. </h3>
  31. > [!NOTE]
  32. > **Contributing: pull requests are by invitation**, and only for changes motivated
  33. > by actual use of Langroid in a real application. To propose a change,
  34. > [open an issue](https://github.com/langroid/langroid/issues/new/choose) naming
  35. > your organization (or that you are independent) and the application that needs
  36. > it. If it fits, we will implement it or invite you to send a PR. For custom work,
  37. > contact Prasad Chalasani (pchalasani at gmail dot com) to discuss terms. See
  38. > [CONTRIBUTING.md](CONTRIBUTING.md) for details.
  39. `Langroid` is an intuitive, lightweight, extensible and principled
  40. Python framework to easily build LLM-powered applications, from CMU and UW-Madison researchers.
  41. You set up Agents, equip them with optional components (LLM,
  42. vector-store and tools/functions), assign them tasks, and have them
  43. collaboratively solve a problem by exchanging messages.
  44. This Multi-Agent paradigm is inspired by the
  45. [Actor Framework](https://en.wikipedia.org/wiki/Actor_model)
  46. (but you do not need to know anything about this!).
  47. `Langroid` is a fresh take on LLM app-development, where considerable thought has gone
  48. into simplifying the developer experience;
  49. it does not use `Langchain`, or any other LLM framework,
  50. and works with [practically any LLM](https://langroid.github.io/langroid/tutorials/supported-models/).
  51. ๐Ÿ”ฅ โœจ A Claude Code [plugin](#claude-code-plugin-optional) is available to
  52. accelerate Langroid development with built-in patterns and best practices.
  53. ๐Ÿ”ฅ Read the (WIP) [overview of the langroid architecture](https://langroid.github.io/langroid/blog/2024/08/15/overview-of-langroids-multi-agent-architecture-prelim/),
  54. and a [quick tour of Langroid](https://langroid.github.io/langroid/tutorials/langroid-tour/).
  55. ๐Ÿ”ฅ MCP Support: Allow any LLM-Agent to leverage MCP Servers via Langroid's simple
  56. [MCP tool adapter](https://langroid.github.io/langroid/notes/mcp-tools/) that converts
  57. the server's tools into Langroid's `ToolMessage` instances.
  58. ๐Ÿ“ข Companies are using/adapting Langroid in **production**. Here is a quote:
  59. >[Nullify](https://www.nullify.ai) uses AI Agents for secure software development.
  60. > It finds, prioritizes and fixes vulnerabilities. We have internally adapted Langroid's multi-agent orchestration framework in production, after evaluating CrewAI, Autogen, LangChain, Langflow, etc. We found Langroid to be far superior to those frameworks in terms of ease of setup and flexibility. Langroid's Agent and Task abstractions are intuitive, well thought out, and provide a great developer experience. We wanted the quickest way to get something in production. With other frameworks it would have taken us weeks, but with Langroid we got to good results in minutes. Highly recommended! <br> -- Jacky Wong, Head of AI at Nullify.
  61. ๐Ÿ”ฅ Just published in ML for Healthcare (2024): a Langroid-based Multi-Agent RAG system for
  62. pharmacovigilance, see [blog post](https://langroid.github.io/langroid/blog/2024/08/12/malade-multi-agent-architecture-for-pharmacovigilance/)
  63. Are you building LLM Applications, or want help with Langroid for your company,
  64. or want to prioritize Langroid features for your company use-cases?
  65. [Prasad Chalasani](https://www.linkedin.com/in/pchalasani/) is available for consulting
  66. (advisory/development): pchalasani at gmail dot com.
  67. Sponsorship is also accepted via [GitHub Sponsors](https://github.com/sponsors/langroid)
  68. **Questions, Feedback, Ideas? Join us on [Discord](https://discord.gg/ZU36McDgDs)!**
  69. # Quick glimpse of coding with Langroid
  70. This is just a teaser; there's much more, like function-calling/tools,
  71. Multi-Agent Collaboration, Structured Information Extraction, DocChatAgent
  72. (RAG), SQLChatAgent, non-OpenAI local/remote LLMs, etc. Scroll down or see docs for more.
  73. See the Langroid Quick-Start [Colab](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb)
  74. that builds up to a 2-agent information-extraction example using the OpenAI ChatCompletion API.
  75. ๐Ÿ”ฅ just released! [Example](https://github.com/langroid/langroid-examples/blob/main/examples/docqa/chat-multi-extract-local.py)
  76. script showing how you can use Langroid multi-agents and tools
  77. to extract structured information from a document using **only a local LLM**
  78. (Mistral-7b-instruct-v0.2).
  79. ```python
  80. import langroid as lr
  81. import langroid.language_models as lm
  82. # set up LLM
  83. llm_cfg = lm.OpenAIGPTConfig(
  84. # any model served via an OpenAI-compatible API
  85. chat_model=lm.OpenAIChatModel.GPT4o, # or, e.g., "ollama/mistral"
  86. )
  87. # use LLM directly
  88. mdl = lm.OpenAIGPT(llm_cfg)
  89. response = mdl.chat("What is the capital of Ontario?", max_tokens=10)
  90. # use LLM in an Agent
  91. agent_cfg = lr.ChatAgentConfig(llm=llm_cfg)
  92. agent = lr.ChatAgent(agent_cfg)
  93. agent.llm_response("What is the capital of China?")
  94. response = agent.llm_response("And India?") # maintains conversation state
  95. # wrap Agent in a Task to run interactive loop with user (or other agents)
  96. task = lr.Task(agent, name="Bot", system_message="You are a helpful assistant")
  97. task.run("Hello") # kick off with user saying "Hello"
  98. # 2-Agent chat loop: Teacher Agent asks questions to Student Agent
  99. teacher_agent = lr.ChatAgent(agent_cfg)
  100. teacher_task = lr.Task(
  101. teacher_agent, name="Teacher",
  102. system_message="""
  103. Ask your student concise numbers questions, and give feedback.
  104. Start with a question.
  105. """
  106. )
  107. student_agent = lr.ChatAgent(agent_cfg)
  108. student_task = lr.Task(
  109. student_agent, name="Student",
  110. system_message="Concisely answer the teacher's questions.",
  111. single_round=True,
  112. )
  113. teacher_task.add_sub_task(student_task)
  114. teacher_task.run()
  115. ```
  116. # ๐Ÿ”ฅ Updates/Releases
  117. <details>
  118. <summary> <b>Click to expand</b></summary>
  119. - **Sep 2026:**
  120. - **[0.68.0](https://github.com/langroid/langroid/releases/tag/0.68.0):** **Removed `OpenAIAssistant`.**
  121. OpenAI [sunset the Assistants API beta on 2026-08-26](https://community.openai.com/t/assistants-api-beta-deprecation-august-26-2026-sunset/1354666);
  122. every endpoint now returns HTTP 404, so the class could not function. It has been
  123. removed along with its tests and examples. `OpenAIGPT` -- which nearly all Langroid
  124. code uses -- is unaffected. See the
  125. [migration note](https://langroid.github.io/langroid/notes/openai-assistant-removal/)
  126. for equivalents to threads, `file_search`, and `code_interpreter`.
  127. - **Aug 2026:**
  128. - [0.67.0](https://github.com/langroid/langroid/releases/tag/0.67.0) Security hardening:
  129. per-provider `env_prefix` for vector-store configs (env-var naming change -- see
  130. [migration notes](https://langroid.github.io/langroid/notes/vecstore-env-prefix/)),
  131. generalized taint propagation across tool re-emission paths, and a one-time warning
  132. when `FileAttachment` payloads inflate context preflight.
  133. - [0.66.0](https://github.com/langroid/langroid/releases/tag/0.66.0) Big community batch (14 PRs):
  134. [Milvus vector store](https://langroid.github.io/langroid/notes/milvus/) (thanks @zc277584121);
  135. [Markdown/HTML document parsing](https://langroid.github.io/langroid/notes/markdown-html-parsing/)
  136. (thanks @nuthalapativarun); cooperative `max_time`
  137. [task budgets](https://langroid.github.io/langroid/notes/task-termination/), MCP tool
  138. namespacing for multi-server agents, and portable JSON chat-history snapshots
  139. (thanks @Whxuan0701); video attachments (thanks @octo-patch); retrieval score
  140. thresholds (thanks @Koushik-Salammagari); even
  141. [context-overflow truncation](https://langroid.github.io/langroid/notes/context-overflow/)
  142. and several routing/parsing fixes -- full details in the
  143. [release notes](https://github.com/langroid/langroid/releases/tag/0.66.0).
  144. - **Aug 2025:**
  145. - [0.59.0](https://github.com/langroid/langroid/releases/tag/0.59.0) Complete Pydantic V2 Migration -
  146. 5-50x faster validation, modern Python patterns, 100% backward compatible.
  147. - **Jul 2025:**
  148. - [0.58.0](https://github.com/langroid/langroid/releases/tag/0.58.0) Crawl4AI integration -
  149. browser-based web crawling with Playwright for JavaScript-heavy sites, no API key required (thank you @abab-dev!).
  150. - [0.57.0](https://github.com/langroid/langroid/releases/tag/0.57.0) HTML Logger for interactive task visualization -
  151. self-contained HTML logs with collapsible entries, auto-refresh, and persistent UI state.
  152. - **Jun 2025:**
  153. - [0.56.0](https://github.com/langroid/langroid/releases/tag/0.56.0) `TaskTool` for delegating tasks to sub-agents -
  154. enables agents to spawn sub-agents with specific tools and configurations.
  155. - [0.55.0](https://github.com/langroid/langroid/releases/tag/0.55.0) Event-based task termination with `done_sequences` -
  156. declarative task completion using event patterns.
  157. - [0.54.0](https://github.com/langroid/langroid/releases/tag/0.54.0) Portkey AI Gateway support - access 200+ models
  158. across providers through unified API with caching, retries, observability.
  159. - **Mar-Apr 2025:**
  160. - [0.53.0](https://github.com/langroid/langroid/releases/tag/0.53.0) MCP Tools Support.
  161. - [0.52.0](https://github.com/langroid/langroid/releases/tag/0.52.0) Multimodal support, i.e. allow PDF, image
  162. inputs to LLM.
  163. - [0.51.0](https://github.com/langroid/langroid/releases/tag/0.51.0) `LLMPdfParser`, generalizing
  164. `GeminiPdfParser` to parse documents directly with LLM.
  165. - [0.50.0](https://github.com/langroid/langroid/releases/tag/0.50.0) Structure-aware Markdown chunking with chunks
  166. enriched by section headers.
  167. - [0.49.0](https://github.com/langroid/langroid/releases/tag/0.49.0) Enable easy switch to LiteLLM Proxy-server
  168. - [0.48.0](https://github.com/langroid/langroid/releases/tag/0.48.0) Exa Crawler, Markitdown Parser
  169. - [0.47.0](https://github.com/langroid/langroid/releases/tag/0.47.0) Support Firecrawl URL scraper/crawler -
  170. thanks @abab-dev
  171. - [0.46.0](https://github.com/langroid/langroid/releases/tag/0.46.0) Support LangDB LLM Gateway - thanks @MrunmayS.
  172. - [0.45.0](https://github.com/langroid/langroid/releases/tag/0.45.0) Markdown parsing with `Marker` - thanks @abab-dev
  173. - [0.44.0](https://github.com/langroid/langroid/releases/tag/0.44.0) Late imports to reduce startup time. Thanks
  174. @abab-dev
  175. - **Feb 2025:**
  176. - [0.43.0](https://github.com/langroid/langroid/releases/tag/0.43.0): `GeminiPdfParser` for parsing PDF using
  177. Gemini LLMs - Thanks @abab-dev.
  178. - [0.42.0](https://github.com/langroid/langroid/releases/tag/0.42.0): `markitdown` parser for `pptx,xlsx,xls` files
  179. Thanks @abab-dev.
  180. - [0.41.0](https://github.com/langroid/langroid/releases/tag/0.41.0): `pinecone` vector-db (Thanks @coretado),
  181. `Tavily` web-search (Thanks @Sozhan308), `Exa` web-search (Thanks @MuddyHope).
  182. - [0.40.0](https://github.com/langroid/langroid/releases/tag/0.40.0): `pgvector` vector-db. Thanks @abab-dev.
  183. - [0.39.0](https://github.com/langroid/langroid/releases/tag/0.39.0): `ChatAgentConfig.handle_llm_no_tool` for
  184. handling LLM "forgetting" to use a tool.
  185. - [0.38.0](https://github.com/langroid/langroid/releases/tag/0.38.0): Gemini embeddings - Thanks @abab-dev)
  186. - [0.37.0](https://github.com/langroid/langroid/releases/tag/0.37.0): New PDF Parsers: `docling`, `pymupdf4llm`
  187. - **Jan 2025:**
  188. - [0.36.0](https://github.com/langroid/langroid/releases/tag/0.36.0): Weaviate vector-db support (thanks @abab-dev).
  189. - [0.35.0](https://github.com/langroid/langroid/releases/tag/0.35.0): Capture/Stream reasoning content from
  190. Reasoning LLMs (e.g. DeepSeek-R1, OpenAI o1) in addition to final answer.
  191. - [0.34.0](https://github.com/langroid/langroid/releases/tag/0.34.0): DocChatAgent
  192. chunk enrichment to improve retrieval. (collaboration with @dfm88).
  193. - [0.33.0](https://github.com/langroid/langroid/releases/tag/0.33.3) Move from Poetry to uv! (thanks @abab-dev).
  194. - [0.32.0](https://github.com/langroid/langroid/releases/tag/0.32.0) DeepSeek v3 support.
  195. - **Dec 2024:**
  196. - [0.31.0](https://github.com/langroid/langroid/releases/tag/0.31.0) Azure OpenAI Embeddings
  197. - [0.30.0](https://github.com/langroid/langroid/releases/tag/0.30.0) Llama-cpp embeddings (thanks @Kwigg).
  198. - [0.29.0](https://github.com/langroid/langroid/releases/tag/0.29.0) Custom Azure OpenAI Client (thanks
  199. @johannestang).
  200. - [0.28.0](https://github.com/langroid/langroid/releases/tag/0.28.0) `ToolMessage`: `_handler` field to override
  201. default handler method name in `request` field (thanks @alexagr).
  202. - [0.27.0](https://github.com/langroid/langroid/releases/tag/0.27.0) OpenRouter Support.
  203. - [0.26.0](https://github.com/langroid/langroid/releases/tag/0.26.0) Update to latest Chainlit.
  204. - [0.25.0](https://github.com/langroid/langroid/releases/tag/0.25.0) True Async Methods for agent and
  205. user-response (thanks @alexagr).
  206. - **Nov 2024:**
  207. - **[0.24.0](https://langroid.github.io/langroid/notes/structured-output/)**:
  208. Enables support for `Agent`s with strict JSON schema output format on compatible LLMs and strict mode for the OpenAI tools API.
  209. (thanks @nilspalumbo).
  210. - **[0.23.0](https://langroid.github.io/langroid/tutorials/local-llm-setup/#local-llms-hosted-on-glhfchat)**:
  211. support for LLMs (e.g. `Qwen2.5-Coder-32b-Instruct`) hosted on glhf.chat
  212. - **[0.22.0](https://langroid.github.io/langroid/notes/large-tool-results/)**:
  213. Optional parameters to truncate large tool results.
  214. - **[0.21.0](https://langroid.github.io/langroid/notes/gemini/)** Direct support for Gemini models via OpenAI client instead of using LiteLLM.
  215. - **[0.20.0](https://github.com/langroid/langroid/releases/tag/0.20.0)** Support for
  216. ArangoDB Knowledge Graphs.
  217. - **Oct 2024:**
  218. - **[0.18.0]** [LLMConfig.async_stream_quiet](https://langroid.github.io/langroid/notes/async-streaming/) flag to
  219. turn off LLM output in async + stream mode.
  220. - **[0.17.0]** XML-based tools, see [docs](https://langroid.github.io/langroid/notes/xml-tools/).
  221. - **Sep 2024:**
  222. - **[0.16.0](https://github.com/langroid/langroid/releases/tag/0.16.0)** Support for OpenAI `o1-mini` and `o1-preview` models.
  223. - **[0.15.0](https://github.com/langroid/langroid/releases/tag/0.15.0)** Cerebras API support -- run llama-3.1 models hosted on Cerebras Cloud (very fast inference).
  224. - **[0.14.0](https://github.com/langroid/langroid/releases/tag/0.14.0)** `DocChatAgent` uses Reciprocal Rank Fusion (RRF) to rank chunks retrieved by different methods.
  225. - **[0.12.0](https://github.com/langroid/langroid/releases/tag/0.12.0)** `run_batch_task` new option -- `stop_on_first_result` - allows termination of batch as soon as any task returns a result.
  226. - **Aug 2024:**
  227. - **[0.11.0](https://github.com/langroid/langroid/releases/tag/0.11.0)** Polymorphic `Task.run(), Task.run_async`.
  228. - **[0.10.0](https://github.com/langroid/langroid/releases/tag/0.10.0)** Allow tool handlers to return arbitrary result type, including other tools.
  229. - **[0.9.0](https://github.com/langroid/langroid/releases/tag/0.9.0)** Orchestration Tools, to signal various task statuses, and to pass messages between agents.
  230. - **[0.7.0](https://github.com/langroid/langroid/releases/tag/0.7.0)** OpenAI tools API support, including multi-tools.
  231. - **Jul 2024:**
  232. - **[0.3.0](https://github.com/langroid/langroid/releases/tag/0.3.0)**: Added [FastEmbed](https://qdrant.github.io/fastembed/qdrant/Usage_With_Qdrant/) embeddings from Qdrant
  233. - **Jun 2024:**
  234. - **0.2.0:** Improved lineage tracking, granular sub-task configs, and a new tool, `RewindTool`,
  235. that lets an agent "rewind and redo" a past message (and all dependent messages are cleared out
  236. thanks to the lineage tracking). Read notes [here](https://github.com/langroid/langroid/releases/tag/0.2.0).
  237. - **May 2024:**
  238. - **Slimmer langroid**: All document-parsers (i.e. pdf, doc, docx) and most
  239. vector-databases (except qdrant)
  240. are now optional/extra dependencies, which helps reduce build size, script
  241. start-up time, and install time. For convenience various grouping of "extras" are
  242. provided, e.g. `doc-chat`, `db` (for database-related dependencies). See updated
  243. install instructions below and in the docs.
  244. - **Few-shot examples** for tools: when defining a [ToolMessage](https://langroid.github.io/langroid/quick-start/chat-agent-tool/#example-find-the-smallest-number-in-a-list), previously you were able to include a classmethod named `examples`,
  245. and a random example from this list would be used to generate a 1-shot example
  246. for the LLM. This has been improved so you can now supply a list of examples
  247. where each example is either a tool instance, or a tuple of (description,
  248. tool instance), where the description is a "thought" that leads the LLM to use
  249. the tool (see example in the [docs](https://langroid.github.io/langroid/quick-start/chat-agent-tool/#example-find-the-smallest-number-in-a-list)). In some scenarios this can improve LLM tool
  250. generation accuracy. Also, now instead of a random example, ALL examples are used to generate few-shot
  251. examples.
  252. - [Infinite loop detection](https://github.com/langroid/langroid/blob/0ed30eb467b00d5eaf2933b577a4b2cc37de1aa1/langroid/agent/task.py#L1121) for task loops of cycle-length <= 10 (configurable
  253. in [`TaskConfig`](https://langroid.github.io/langroid/reference/agent/task/#langroid.agent.task.TaskConfig). Only detects _exact_ loops, rather than _approximate_ loops where the entities are saying essentially similar (but not exactly the same) things repeatedly.
  254. - "@"-addressing: any entity can address any other by name, which can be the name
  255. of an agent's responder ("llm", "user", "agent") or a sub-task name. This is a
  256. simpler alternative to the `RecipientTool` mechanism, with the tradeoff that
  257. since it's not a tool, there's no way to enforce/remind the LLM to explicitly
  258. specify an addressee (in scenarios where this is important).
  259. - [Much-Improved Citation](https://github.com/langroid/langroid/issues/477)
  260. generation and display when using `DocChatAgent`.
  261. - `gpt-4o` is now the default LLM throughout; Update tests and examples to work
  262. with this LLM; use tokenizer corresponding to the LLM.
  263. - `gemini 1.5 pro` support via `litellm`
  264. - `QdrantDB:` update to support learned sparse embeddings.
  265. - **Apr 2024:**
  266. - **0.1.236**: Support for open LLMs hosted on Groq, e.g. specify
  267. `chat_model="groq/llama3-8b-8192"`.
  268. See [tutorial](https://langroid.github.io/langroid/tutorials/local-llm-setup/).
  269. - **0.1.235**: `Task.run(), Task.run_async(), run_batch_tasks` have `max_cost`
  270. and `max_tokens` params to exit when tokens or cost exceed a limit. The result
  271. `ChatDocument.metadata` now includes a `status` field which is a code indicating a
  272. task completion reason code. Also `task.run()` etc can be invoked with an explicit
  273. `session_id` field which is used as a key to look up various settings in Redis cache.
  274. Currently only used to look up "kill status" - this allows killing a running task, either by `task.kill()`
  275. or by the classmethod `Task.kill_session(session_id)`.
  276. For example usage, see the `test_task_kill` in [tests/main/test_task.py](https://github.com/langroid/langroid/blob/main/tests/main/test_task.py)
  277. - **Mar 2024:**
  278. - **0.1.216:** Improvements to allow concurrent runs of `DocChatAgent`, see the
  279. [`test_doc_chat_agent.py`](https://github.com/langroid/langroid/blob/main/tests/main/test_doc_chat_agent.py)
  280. in particular the `test_doc_chat_batch()`;
  281. New task run utility: [`run_batch_task_gen`](https://github.com/langroid/langroid/blob/main/langroid/agent/batch.py)
  282. where a task generator can be specified, to generate one task per input.
  283. - **0.1.212:** ImagePdfParser: support for extracting text from image-based PDFs.
  284. (this means `DocChatAgent` will now work with image-pdfs).
  285. - **0.1.194 - 0.1.211:** Misc fixes, improvements, and features:
  286. - Big enhancement in RAG performance (mainly, recall) due to a [fix in Relevance
  287. Extractor](https://github.com/langroid/langroid/releases/tag/0.1.209)
  288. - `DocChatAgent` [context-window fixes](https://github.com/langroid/langroid/releases/tag/0.1.208)
  289. - Anthropic/Claude3 support via Litellm
  290. - `URLLoader`: detect file time from header when URL doesn't end with a
  291. recognizable suffix like `.pdf`, `.docx`, etc.
  292. - Misc lancedb integration fixes
  293. - Auto-select embedding config based on whether `sentence_transformer` module is available.
  294. - Slim down dependencies, make some heavy ones optional, e.g. `unstructured`,
  295. `haystack`, `chromadb`, `mkdocs`, `huggingface-hub`, `sentence-transformers`.
  296. - Easier top-level imports from `import langroid as lr`
  297. - Improve JSON detection, esp from weak LLMs
  298. - **Feb 2024:**
  299. - **0.1.193:** Support local LLMs using Ollama's new OpenAI-Compatible server:
  300. simply specify `chat_model="ollama/mistral"`. See [release notes](https://github.com/langroid/langroid/releases/tag/0.1.193).
  301. - **0.1.183:** Added Chainlit support via [callbacks](https://github.com/langroid/langroid/blob/main/langroid/agent/callbacks/chainlit.py).
  302. See [examples](https://github.com/langroid/langroid/tree/main/examples/chainlit).
  303. - **Jan 2024:**
  304. - **0.1.175**
  305. - [Neo4jChatAgent](https://github.com/langroid/langroid/tree/main/langroid/agent/special/neo4j) to chat with a neo4j knowledge-graph.
  306. (Thanks to [Mohannad](https://github.com/Mohannadcse)!). The agent uses tools to query the Neo4j schema and translate user queries to Cypher queries,
  307. and the tool handler executes these queries, returning them to the LLM to compose
  308. a natural language response (analogous to how `SQLChatAgent` works).
  309. See example [script](https://github.com/langroid/langroid/tree/main/examples/kg-chat) using this Agent to answer questions about Python pkg dependencies.
  310. - Support for `.doc` file parsing (in addition to `.docx`)
  311. - Specify optional [`formatter` param](https://github.com/langroid/langroid/releases/tag/0.1.171)
  312. in `OpenAIGPTConfig` to ensure accurate chat formatting for local LLMs.
  313. - **[0.1.157](https://github.com/langroid/langroid/releases/tag/0.1.157):** `DocChatAgentConfig`
  314. has a new param: `add_fields_to_content`, to specify additional document fields to insert into
  315. the main `content` field, to help improve retrieval.
  316. - **[0.1.156](https://github.com/langroid/langroid/releases/tag/0.1.156):** New Task control signals
  317. PASS_TO, SEND_TO; VectorStore: Compute Pandas expression on documents; LanceRAGTaskCreator creates 3-agent RAG system with Query Planner, Critic and RAG Agent.
  318. - **Dec 2023:**
  319. - **0.1.154:** (For details see release notes of [0.1.149](https://github.com/langroid/langroid/releases/tag/0.1.149)
  320. and [0.1.154](https://github.com/langroid/langroid/releases/tag/0.1.154)).
  321. - `DocChatAgent`: Ingest Pandas dataframes and filtering.
  322. - `LanceDocChatAgent` leverages `LanceDB` vector-db for efficient vector search
  323. and full-text search and filtering.
  324. - Improved task and multi-agent control mechanisms
  325. - `LanceRAGTaskCreator` to create a 2-agent system consisting of a `LanceFilterAgent` that
  326. decides a filter and rephrase query to send to a RAG agent.
  327. - **[0.1.141](https://github.com/langroid/langroid/releases/tag/0.1.141):**
  328. API Simplifications to reduce boilerplate:
  329. auto-select an available OpenAI model (preferring gpt-4o), simplifies defaults.
  330. Simpler `Task` initialization with default `ChatAgent`.
  331. - **Nov 2023:**
  332. > **Note:** the `OpenAIAssistant` entries below are historical. The class was
  333. > removed in 0.68.0 after OpenAI sunset the Assistants API; the links point at
  334. > the 0.67.5 tag. See the
  335. > [migration note](https://langroid.github.io/langroid/notes/openai-assistant-removal/).
  336. - **[0.1.126](https://github.com/langroid/langroid/releases/tag/0.1.126):**
  337. OpenAIAssistant agent: Caching Support.
  338. - **0.1.117:** Support for OpenAI Assistant API tools: Function-calling,
  339. Code-intepreter, and Retriever (RAG), file uploads. These work seamlessly
  340. with Langroid's task-orchestration.
  341. Until docs are ready, it's best to see these usage examples:
  342. - **Tests:**
  343. - [test_openai_assistant.py](https://github.com/langroid/langroid/blob/0.67.5/tests/main/test_openai_assistant.py)
  344. - [test_openai_assistant_async.py](https://github.com/langroid/langroid/blob/0.67.5/tests/main/test_openai_assistant_async.py)
  345. - **Example scripts:**
  346. - [The most basic chat app](https://github.com/langroid/langroid/blob/0.67.5/examples/basic/oai-asst-chat.py)
  347. - [Chat with code interpreter](https://github.com/langroid/langroid/blob/0.67.5/examples/basic/oai-code-chat.py)
  348. - [Chat with retrieval (RAG)](https://github.com/langroid/langroid/blob/0.67.5/examples/docqa/oai-retrieval-assistant.py)
  349. - [2-agent RAG chat](https://github.com/langroid/langroid/blob/0.67.5/examples/docqa/oai-retrieval-2.py)
  350. - **0.1.112:** [`OpenAIAssistant`](https://github.com/langroid/langroid/blob/0.67.5/langroid/agent/openai_assistant.py) is a subclass of `ChatAgent` that
  351. leverages the new OpenAI Assistant API. It can be used as a drop-in
  352. replacement for `ChatAgent`, and relies on the Assistant API to
  353. maintain conversation state, and leverages persistent threads and
  354. assistants to reconnect to them if needed. Examples:
  355. [`test_openai_assistant.py`](https://github.com/langroid/langroid/blob/0.67.5/tests/main/test_openai_assistant.py),
  356. [`test_openai_assistant_async.py`](https://github.com/langroid/langroid/blob/0.67.5/tests/main/test_openai_assistant_async.py)
  357. - **0.1.111:** Support latest OpenAI model: `GPT4_TURBO`
  358. (see [test_llm.py](https://github.com/langroid/langroid/blob/main/tests/main/test_llm.py) for example usage)
  359. - **0.1.110:** Upgrade from OpenAI v0.x to v1.1.1 (in preparation for
  360. Assistants API and more); (`litellm` temporarily disabled due to OpenAI
  361. version conflict).
  362. - **Oct 2023:**
  363. - **0.1.107:** `DocChatAgent` re-rankers: `rank_with_diversity`, `rank_to_periphery` (lost in middle).
  364. - **0.1.102:** `DocChatAgentConfig.n_neighbor_chunks > 0` allows returning context chunks around match.
  365. - **0.1.101:** `DocChatAgent` uses `RelevanceExtractorAgent` to have
  366. the LLM extract relevant portions of a chunk using
  367. sentence-numbering, resulting in huge speed up and cost reduction
  368. compared to the naive "sentence-parroting" approach (writing out full
  369. sentences out relevant whole sentences) which `LangChain` uses in their
  370. `LLMChainExtractor`.
  371. - **0.1.100:** API update: all of Langroid is accessible with a single import, i.e. `import langroid as lr`. See the [documentation]("https://langroid.github.io/langroid/") for usage.
  372. - **0.1.99:** Convenience batch functions to run tasks, agent methods on a list of inputs concurrently in async mode. See examples in [test_batch.py](https://github.com/langroid/langroid/blob/main/tests/main/test_batch.py).
  373. - **0.1.95:** Added support for [Momento Serverless Vector Index](https://docs.momentohq.com/vector-index)
  374. - **0.1.94:** Added support for [LanceDB](https://lancedb.github.io/lancedb/) vector-store -- allows vector, Full-text, SQL search.
  375. - **0.1.84:** Added [LiteLLM](https://docs.litellm.ai/docs/providers), so now Langroid can be used with over 100 LLM providers (remote or local)!
  376. See guide [here](https://langroid.github.io/langroid/tutorials/non-openai-llms/).
  377. - **Sep 2023:**
  378. - **0.1.78:** Async versions of several Task, Agent and LLM methods;
  379. Nested Pydantic classes are now supported for LLM Function-calling, Tools, Structured Output.
  380. - **0.1.76:** DocChatAgent: support for loading `docx` files (preliminary).
  381. - **0.1.72:** Many improvements to DocChatAgent: better embedding model,
  382. hybrid search to improve retrieval, better pdf parsing, re-ranking retrieved results with cross-encoders.
  383. - **Use with local LLama Models:** see tutorial [here](https://langroid.github.io/langroid/blog/2023/09/14/using-langroid-with-local-llms/)
  384. - **Langroid Blog/Newsletter Launched!**: First post is [here](https://substack.com/notes/post/p-136704592) -- Please subscribe to stay updated.
  385. - **0.1.56:** Support Azure OpenAI.
  386. - **0.1.55:** Improved [`SQLChatAgent`](https://github.com/langroid/langroid/blob/main/langroid/agent/special/sql/sql_chat_agent.py) that efficiently retrieves relevant schema info when translating natural language to SQL.
  387. - **Aug 2023:**
  388. - **[Hierarchical computation](https://langroid.github.io/langroid/examples/agent-tree/)** example using Langroid agents and task orchestration.
  389. - **0.1.51:** Support for global state, see [test_global_state.py](https://github.com/langroid/langroid/blob/main/tests/main/test_global_state.py).
  390. - **๐Ÿณ Langroid Docker image**, available, see instructions below.
  391. - [**RecipientTool**](https://github.com/langroid/langroid/blob/main/langroid/agent/tools/recipient_tool.py) enables (+ enforces) LLM to
  392. specify an intended recipient when talking to 2 or more agents.
  393. See [this test](https://github.com/langroid/langroid/blob/main/tests/main/test_recipient_tool.py) for example usage.
  394. - **Example:** [Answer questions](https://github.com/langroid/langroid/blob/main/examples/docqa/chat-search.py) using Google Search + vecdb-retrieval from URL contents.
  395. - **0.1.39:** [`GoogleSearchTool`](https://github.com/langroid/langroid/blob/main/langroid/agent/tools/google_search_tool.py) to enable Agents (their LLM) to do Google searches via function-calling/tools.
  396. See [this chat example](https://github.com/langroid/langroid/blob/main/examples/basic/chat-search.py) for how easy it is to add this tool to an agent.
  397. - **Colab notebook** to try the quick-start examples: [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb)
  398. - **0.1.37:** Added [`SQLChatAgent`](https://github.com/langroid/langroid/blob/main/langroid/agent/special/sql_chat_agent.py) -- thanks to our latest contributor [Rithwik Babu](https://github.com/rithwikbabu)!
  399. - Multi-agent Example: [Autocorrect chat](https://github.com/langroid/langroid/blob/main/examples/basic/autocorrect.py)
  400. - **July 2023:**
  401. - **0.1.30:** Added [`TableChatAgent`](https://github.com/langroid/langroid/blob/main/langroid/agent/special/table_chat_agent.py) to
  402. [chat](https://github.com/langroid/langroid/blob/main/examples/data-qa/table_chat.py) with tabular datasets (dataframes, files, URLs): LLM generates Pandas code,
  403. and code is executed using Langroid's tool/function-call mechanism.
  404. - **Demo:** 3-agent system for Audience [Targeting](https://langroid.github.io/langroid/demos/targeting/audience-targeting/).
  405. - **0.1.27**: Added [support](https://github.com/langroid/langroid/blob/main/langroid/cachedb/momento_cachedb.py)
  406. for [Momento Serverless Cache](https://www.gomomento.com/) as an alternative to Redis.
  407. - **0.1.24**: [`DocChatAgent`](https://github.com/langroid/langroid/blob/main/langroid/agent/special/doc_chat_agent.py)
  408. now [accepts](https://github.com/langroid/langroid/blob/main/langroid/parsing/document_parser.py) PDF files or URLs.
  409. </details>
  410. # ๐Ÿš€ Demo
  411. Suppose you want to extract structured information about the key terms
  412. of a commercial lease document. You can easily do this with Langroid using a two-agent system,
  413. as we show in the [langroid-examples](https://github.com/langroid/langroid-examples/blob/main/examples/docqa/chat_multi_extract.py) repo.
  414. (See [this script](https://github.com/langroid/langroid-examples/blob/main/examples/docqa/chat-multi-extract-local.py)
  415. for a version with the same functionality using a local Mistral-7b model.)
  416. The demo showcases just a few of the many features of Langroid, such as:
  417. - Multi-agent collaboration: `LeaseExtractor` is in charge of the task, and its LLM (GPT4) generates questions
  418. to be answered by the `DocAgent`.
  419. - Retrieval augmented question-answering, with **source-citation**: `DocAgent` LLM (GPT4) uses retrieval from a vector-store to
  420. answer the `LeaseExtractor`'s questions, cites the specific excerpt supporting the answer.
  421. - Function-calling (also known as tool/plugin): When it has all the information it
  422. needs, the `LeaseExtractor` LLM presents the information in a structured
  423. format using a Function-call.
  424. Here is what it looks like in action
  425. (a pausable mp4 video is [here](https://vimeo.com/871429249)).
  426. ![Demo](https://raw.githubusercontent.com/langroid/langroid/main/docs/assets/demos/lease-extractor-demo.gif)
  427. # โšก Highlights
  428. (For a more up-to-date list see the
  429. [Updates/Releases](https://github.com/langroid/langroid?tab=readme-ov-file#-updatesreleases)
  430. section above)
  431. - **Agents as first-class citizens:** The [Agent](https://langroid.github.io/langroid/reference/agent/base/#langroid.agent.base.Agent) class encapsulates LLM conversation state,
  432. and optionally a vector-store and tools. Agents are a core abstraction in Langroid;
  433. Agents act as _message transformers_, and by default provide 3 _responder_ methods, one corresponding to each entity: LLM, Agent, User.
  434. - **Tasks:** A [Task](https://langroid.github.io/langroid/reference/agent/task/) class wraps an Agent, and gives the agent instructions (or roles, or goals),
  435. manages iteration over an Agent's responder methods,
  436. and orchestrates multi-agent interactions via hierarchical, recursive
  437. task-delegation. The `Task.run()` method has the same
  438. type-signature as an Agent's responder's methods, and this is key to how
  439. a task of an agent can delegate to other sub-tasks: from the point of view of a Task,
  440. sub-tasks are simply additional responders, to be used in a round-robin fashion
  441. after the agent's own responders.
  442. - **Modularity, Reusability, Loose coupling:** The `Agent` and `Task` abstractions allow users to design
  443. Agents with specific skills, wrap them in Tasks, and combine tasks in a flexible way.
  444. - **LLM Support**: Langroid supports OpenAI LLMs as well as LLMs from hundreds of
  445. providers ([local/open](https://langroid.github.io/langroid/tutorials/local-llm-setup/) or [remote/commercial](https://langroid.github.io/langroid/tutorials/non-openai-llms/)) via proxy libraries and local model servers
  446. such as [ollama](https://github.com/ollama), [oobabooga](https://github.com/oobabooga/text-generation-webui),
  447. [LiteLLM](https://docs.litellm.ai/docs/providers) that in effect mimic the OpenAI API. See the [supported LLMs](https://langroid.github.io/langroid/tutorials/supported-models/).
  448. - **Caching of LLM responses:** Langroid supports [Redis](https://redis.com/try-free/) to cache LLM responses.
  449. - **Vector-stores**: [Qdrant](https://qdrant.tech/), [Chroma](https://www.trychroma.com/), LanceDB, Pinecone, PostgresDB (PGVector), Weaviate are currently supported.
  450. Vector stores allow for Retrieval-Augmented-Generation (RAG).
  451. - **Grounding and source-citation:** Access to external documents via vector-stores
  452. allows for grounding and source-citation.
  453. - **Observability, Logging, Lineage:** Langroid generates detailed logs of multi-agent interactions and
  454. maintains provenance/lineage of messages, so that you can trace back
  455. the origin of a message.
  456. - **[Tools/Plugins/Function-calling](https://langroid.github.io/langroid/quick-start/chat-agent-tool/)**:
  457. Langroid supports OpenAI's [function calling](https://platform.openai.com/docs/guides/gpt/function-calling), as
  458. well as an equivalent `ToolMessage` mechanism which works with
  459. any LLM, not just OpenAI's.
  460. Function calling and tools have the same developer-facing interface, implemented
  461. using [Pydantic](https://docs.pydantic.dev/latest/),
  462. which makes it very easy to define tools/functions and enable agents
  463. to use them. Benefits of using Pydantic are that you never have to write
  464. complex JSON specs for function calling, and when the LLM
  465. hallucinates malformed JSON, the Pydantic error message is sent back to
  466. the LLM so it can fix it.
  467. ---
  468. # โš™๏ธ Installation and Setup
  469. ### Install `langroid`
  470. Langroid requires Python 3.11+. We recommend using a virtual environment.
  471. Use `pip` to install a bare-bones slim version of `langroid` (from PyPi) to your virtual
  472. environment:
  473. ```bash
  474. pip install langroid
  475. ```
  476. The core Langroid package lets you use OpenAI Embeddings models via their API.
  477. If you instead want to use the `sentence-transformers` embedding models from HuggingFace,
  478. install Langroid like this:
  479. ```bash
  480. pip install "langroid[hf-embeddings]"
  481. ```
  482. For many practical scenarios, you may need additional optional dependencies:
  483. - To use various document-parsers, install langroid with the `doc-chat` extra:
  484. ```bash
  485. pip install "langroid[doc-chat]"
  486. ```
  487. - For "chat with databases", use the `db` extra:
  488. ```bash
  489. pip install "langroid[db]"
  490. ```
  491. - You can specify multiple extras by separating them with commas, e.g.:
  492. ```bash
  493. pip install "langroid[doc-chat,db]"
  494. ```
  495. - To simply install _all_ optional dependencies, use the `all` extra (but note that this will result in longer load/startup times and a larger install size):
  496. ```bash
  497. pip install "langroid[all]"
  498. ```
  499. <details>
  500. <summary><b>Optional Installs for using SQL Chat with a PostgreSQL DB </b></summary>
  501. If you are using `SQLChatAgent`
  502. (e.g. the script [`examples/data-qa/sql-chat/sql_chat.py`](https://github.com/langroid/langroid/blob/main/examples/data-qa/sql-chat/sql_chat.py)),
  503. with a postgres db, you will need to:
  504. - Install PostgreSQL dev libraries for your platform, e.g.
  505. - `sudo apt-get install libpq-dev` on Ubuntu,
  506. - `brew install postgresql` on Mac, etc.
  507. - Install langroid with the postgres extra, e.g. `pip install langroid[postgres]`
  508. or `poetry add "langroid[postgres]"` or `poetry install -E postgres`,
  509. (or the corresponding `uv` versions, e.g. `uv add "langroid[postgres]"`
  510. or `uv pip install langroid[postgres]`).
  511. If this gives you an error, try `pip install psycopg2-binary` in your virtualenv.
  512. </details>
  513. ๐Ÿ“ If you get strange errors involving `mysqlclient`, try doing `pip uninstall mysqlclient` followed by `pip install mysqlclient`.
  514. ### Claude Code Plugin (Optional)
  515. This plugin provides two skills:
  516. - `langroid:patterns` - Your Claude Code agent can leverage this skill to produce
  517. Langroid multi-agent code using proper design patterns.
  518. - `langroid:add-pattern` - The agent can use this skill to record new patterns it
  519. learns, for future reference, either autonomously or when prompted by the user.
  520. **Step 1: Add the Langroid marketplace**
  521. From terminal:
  522. ```bash
  523. claude plugin marketplace add langroid/langroid
  524. ```
  525. Or within Claude Code:
  526. ```
  527. /plugin marketplace add langroid/langroid
  528. ```
  529. **Step 2: Install the Langroid plugin**
  530. From terminal:
  531. ```bash
  532. claude plugin install langroid@langroid
  533. ```
  534. Or within Claude Code:
  535. ```
  536. /plugin install langroid@langroid
  537. ```
  538. Once installed, simply ask your Claude Code agent to implement Langroid patterns in
  539. natural language, e.g.,
  540. > set up a Langroid agent so it uses the EditTool, and wrap it in a task that ends as soon as the tool is generated
  541. and it will automatically use the `langroid:patterns` skill to follow the right design pattern.
  542. You can also ask Claude Code to record a new pattern when you discover one, e.g.,
  543. > record this as a new Langroid pattern for setting up MCP tools
  544. ### Set up environment variables (API keys, etc)
  545. To get started, all you need is an OpenAI API Key.
  546. If you don't have one, see [this OpenAI Page](https://platform.openai.com/docs/quickstart).
  547. (Note that while this is the simplest way to get started, Langroid works with practically any LLM, not just those from OpenAI.
  548. See the guides to using [Open/Local LLMs](https://langroid.github.io/langroid/tutorials/local-llm-setup/),
  549. and other [non-OpenAI](https://langroid.github.io/langroid/tutorials/non-openai-llms/) proprietary LLMs.)
  550. In the root of the repo, copy the `.env-template` file to a new file `.env`:
  551. ```bash
  552. cp .env-template .env
  553. ```
  554. Then insert your OpenAI API Key.
  555. Your `.env` file should look like this (the organization is optional
  556. but may be required in some scenarios).
  557. ```bash
  558. OPENAI_API_KEY=your-key-here-without-quotes
  559. OPENAI_ORGANIZATION=optionally-your-organization-id
  560. ````
  561. Alternatively, you can set this as an environment variable in your shell
  562. (you will need to do this every time you open a new shell):
  563. ```bash
  564. export OPENAI_API_KEY=your-key-here-without-quotes
  565. ```
  566. <details>
  567. <summary><b>Optional Setup Instructions (click to expand) </b></summary>
  568. All of the following environment variable settings are optional, and some are only needed
  569. to use specific features (as noted below).
  570. - **Qdrant** Vector Store API Key, URL. This is only required if you want to use Qdrant cloud.
  571. Alternatively [Chroma](https://docs.trychroma.com/),
  572. [LanceDB](https://lancedb.com/), or [Milvus](https://milvus.io/) are also
  573. currently supported.
  574. We use the local-storage version of Chroma, so there is no need for an API key.
  575. - **Milvus** Vector Store URI, token, database name. These are optional.
  576. Without them, Milvus uses local storage at `./milvus.db`. Set `MILVUS_URI`
  577. for Milvus server or Zilliz Cloud, `MILVUS_TOKEN` for token-based access,
  578. and `MILVUS_DB_NAME` for a named database. Milvus Lite is unavailable on
  579. Windows, so Windows users must set `MILVUS_URI` to a Milvus server or Zilliz
  580. Cloud endpoint.
  581. - **Redis** Password, host, port: This is optional, and only needed to cache LLM API responses
  582. using Redis Cloud. Redis [offers](https://redis.com/try-free/) a free 30MB Redis account
  583. which is more than sufficient to try out Langroid and even beyond.
  584. If you don't set up these, Langroid will use a pure-python
  585. Redis in-memory cache via the [Fakeredis](https://fakeredis.readthedocs.io/en/latest/) library.
  586. - **Momento** Serverless Caching of LLM API responses (as an alternative to Redis).
  587. To use Momento instead of Redis:
  588. - enter your Momento Token in the `.env` file, as the value of `MOMENTO_AUTH_TOKEN` (see example file below),
  589. - in the `.env` file set `CACHE_TYPE=momento` (instead of `CACHE_TYPE=redis` which is the default).
  590. - **GitHub** Personal Access Token (required for apps that need to analyze git
  591. repos; token-based API calls are less rate-limited). See this
  592. [GitHub page](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens).
  593. - **Google Custom Search API Credentials:** Only needed to enable an Agent to use the `GoogleSearchTool`.
  594. To use Google Search as an LLM Tool/Plugin/function-call,
  595. you'll need to set up
  596. [a Google API key](https://developers.google.com/custom-search/v1/introduction#identify_your_application_to_google_with_api_key),
  597. then [setup a Google Custom Search Engine (CSE) and get the CSE ID](https://developers.google.com/custom-search/docs/tutorial/creatingcse).
  598. (Documentation for these can be challenging, we suggest asking GPT4 for a step-by-step guide.)
  599. After obtaining these credentials, store them as values of
  600. `GOOGLE_API_KEY` and `GOOGLE_CSE_ID` in your `.env` file.
  601. Full documentation on using this (and other such "stateless" tools) is coming soon, but
  602. in the meantime take a peek at this [chat example](https://github.com/langroid/langroid/blob/main/examples/basic/chat-search.py), which
  603. shows how you can easily equip an Agent with a `GoogleSearchtool`.
  604. An `.env` file with commonly used optional variables looks like this:
  605. ```bash
  606. OPENAI_API_KEY=your-key-here-without-quotes
  607. GITHUB_ACCESS_TOKEN=your-personal-access-token-no-quotes
  608. CACHE_TYPE=redis # or momento
  609. REDIS_PASSWORD=your-redis-password-no-quotes
  610. REDIS_HOST=your-redis-hostname-no-quotes
  611. REDIS_PORT=your-redis-port-no-quotes
  612. MOMENTO_AUTH_TOKEN=your-momento-token-no-quotes # instead of REDIS* variables
  613. QDRANT_API_KEY=your-key
  614. QDRANT_API_URL=https://your.url.here:6333 # note port number must be included
  615. GOOGLE_API_KEY=your-key
  616. GOOGLE_CSE_ID=your-cse-id
  617. ```
  618. </details>
  619. <details>
  620. <summary><b>Optional setup instructions for Microsoft Azure OpenAI(click to expand)</b></summary>
  621. When using Azure OpenAI, additional environment variables are required in the
  622. `.env` file.
  623. This page [Microsoft Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-services/openai/chatgpt-quickstart?tabs=command-line&pivots=programming-language-python#environment-variables)
  624. provides more information, and you can set each environment variable as follows:
  625. - `AZURE_OPENAI_API_KEY`, from the value of `API_KEY`
  626. - `AZURE_OPENAI_API_BASE` from the value of `ENDPOINT`, typically looks like `https://your.domain.azure.com`.
  627. - For `AZURE_OPENAI_API_VERSION`, you can use the default value in `.env-template`, and latest version can be found [here](https://learn.microsoft.com/en-us/azure/ai-services/openai/whats-new#azure-openai-chat-completion-general-availability-ga)
  628. - `AZURE_OPENAI_DEPLOYMENT_NAME` is the name of the deployed model, which is defined by the user during the model setup
  629. - `AZURE_OPENAI_MODEL_NAME` Azure OpenAI allows specific model names when you select the model for your deployment. You need to put precisly the exact model name that was selected. For example, GPT-4 (should be `gpt-4-32k` or `gpt-4`).
  630. - `AZURE_OPENAI_MODEL_VERSION` is required if `AZURE_OPENAI_MODEL_NAME = gpt=4`, which will assist Langroid to determine the cost of the model
  631. </details>
  632. ---
  633. # ๐Ÿณ Docker Instructions
  634. We provide a containerized version of the [`langroid-examples`](https://github.com/langroid/langroid-examples)
  635. repository via this [Docker Image](https://hub.docker.com/r/langroid/langroid).
  636. All you need to do is set up environment variables in the `.env` file.
  637. Please follow these steps to setup the container:
  638. ```bash
  639. # get the .env file template from `langroid` repo
  640. wget -O .env https://raw.githubusercontent.com/langroid/langroid/main/.env-template
  641. # Edit the .env file with your favorite editor (here nano), and remove any un-used settings. E.g. there are "dummy" values like "your-redis-port" etc -- if you are not using them, you MUST remove them.
  642. nano .env
  643. # launch the container (the appropriate image for your architecture will be pulled automatically)
  644. docker run -it --rm -v ./.env:/langroid/.env langroid/langroid:latest
  645. # Use this command to run any of the scripts in the `examples` directory
  646. python examples/<Path/To/Example.py>
  647. ```
  648. # ๐ŸŽ‰ Usage Examples
  649. These are quick teasers to give a glimpse of what you can do with Langroid
  650. and how your code would look.
  651. โš ๏ธ The code snippets below are intended to give a flavor of the code
  652. and they are **not** complete runnable examples! For that we encourage you to
  653. consult the [`langroid-examples`](https://github.com/langroid/langroid-examples)
  654. repository.
  655. โ„น๏ธ
  656. The various LLM prompts and instructions in Langroid
  657. have been tested to work well with GPT-4 (and to some extent GPT-4o).
  658. Switching to other LLMs (local/open and proprietary) is easy (see guides mentioned above),
  659. and may suffice for some applications, but in general you may see inferior results
  660. unless you adjust the prompts and/or the multi-agent setup.
  661. ๐Ÿ“– Also see the
  662. [`Getting Started Guide`](https://langroid.github.io/langroid/quick-start/)
  663. for a detailed tutorial.
  664. Click to expand any of the code examples below.
  665. All of these can be run in a Colab notebook:
  666. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb)
  667. <details>
  668. <summary> <b> Direct interaction with LLM </b> </summary>
  669. ```python
  670. import langroid.language_models as lm
  671. mdl = lm.OpenAIGPT(
  672. lm.OpenAIGPTConfig(
  673. chat_model=lm.OpenAIChatModel.GPT4o, # or, e.g. "ollama/qwen2.5"
  674. ),
  675. )
  676. messages = [
  677. lm.LLMMessage(content="You are a helpful assistant", role=lm.Role.SYSTEM),
  678. lm.LLMMessage(content="What is the capital of Ontario?", role=lm.Role.USER),
  679. ]
  680. response = mdl.chat(messages, max_tokens=200)
  681. print(response.message)
  682. ```
  683. See the guides to use
  684. ([local/open LLMs](https://langroid.github.io/langroid/tutorials/local-llm-setup/) or [remote/commercial LLMs](https://langroid.github.io/langroid/tutorials/non-openai-llms/)).
  685. </details>
  686. <details>
  687. <summary> <b> Interaction with non-OpenAI LLM (local or remote) </b> </summary>
  688. Local model: if model is served at `http://localhost:8000`:
  689. ```python
  690. cfg = lm.OpenAIGPTConfig(
  691. chat_model="local/localhost:8000",
  692. chat_context_length=4096
  693. )
  694. mdl = lm.OpenAIGPT(cfg)
  695. # now interact with it as above, or create an Agent + Task as shown below.
  696. ```
  697. </details>
  698. <details>
  699. <summary> <b> Define an agent, set up a task, and run it </b> </summary>
  700. ```python
  701. import langroid as lr
  702. agent = lr.ChatAgent()
  703. # get response from agent's LLM, and put this in an interactive loop...
  704. # answer = agent.llm_response("What is the capital of Ontario?")
  705. # ... OR instead, set up a task (which has a built-in loop) and run it
  706. task = lr.Task(agent, name="Bot")
  707. task.run() # ... a loop seeking response from LLM or User at each turn
  708. ```
  709. </details>
  710. <details>
  711. <summary><b> Three communicating agents </b></summary>
  712. A toy numbers game, where when given a number `n`:
  713. - `repeater_task`'s LLM simply returns `n`,
  714. - `even_task`'s LLM returns `n/2` if `n` is even, else says "DO-NOT-KNOW"
  715. - `odd_task`'s LLM returns `3*n+1` if `n` is odd, else says "DO-NOT-KNOW"
  716. Each of these `Task`s automatically configures a default `ChatAgent`.
  717. ```python
  718. import langroid as lr
  719. from langroid.utils.constants import NO_ANSWER
  720. repeater_task = lr.Task(
  721. name = "Repeater",
  722. system_message="""
  723. Your job is to repeat whatever number you receive.
  724. """,
  725. llm_delegate=True, # LLM takes charge of task
  726. single_round=False,
  727. )
  728. even_task = lr.Task(
  729. name = "EvenHandler",
  730. system_message=f"""
  731. You will be given a number.
  732. If it is even, divide by 2 and say the result, nothing else.
  733. If it is odd, say {NO_ANSWER}
  734. """,
  735. single_round=True, # task done after 1 step() with valid response
  736. )
  737. odd_task = lr.Task(
  738. name = "OddHandler",
  739. system_message=f"""
  740. You will be given a number n.
  741. If it is odd, return (n*3+1), say nothing else.
  742. If it is even, say {NO_ANSWER}
  743. """,
  744. single_round=True, # task done after 1 step() with valid response
  745. )
  746. ```
  747. Then add the `even_task` and `odd_task` as sub-tasks of `repeater_task`,
  748. and run the `repeater_task`, kicking it off with a number as input:
  749. ```python
  750. repeater_task.add_sub_task([even_task, odd_task])
  751. repeater_task.run("3")
  752. ```
  753. </details>
  754. <details>
  755. <summary><b> Simple Tool/Function-calling example </b></summary>
  756. Langroid leverages Pydantic to support OpenAI's
  757. [Function-calling API](https://platform.openai.com/docs/guides/gpt/function-calling)
  758. as well as its own native tools. The benefits are that you don't have to write
  759. any JSON to specify the schema, and also if the LLM hallucinates a malformed
  760. tool syntax, Langroid sends the Pydantic validation error (suitably sanitized)
  761. to the LLM so it can fix it!
  762. Simple example: Say the agent has a secret list of numbers,
  763. and we want the LLM to find the smallest number in the list.
  764. We want to give the LLM a `probe` tool/function which takes a
  765. single number `n` as argument. The tool handler method in the agent
  766. returns how many numbers in its list are at most `n`.
  767. First define the tool using Langroid's `ToolMessage` class:
  768. ```python
  769. import langroid as lr
  770. class ProbeTool(lr.agent.ToolMessage):
  771. request: str = "probe" # specifies which agent method handles this tool
  772. purpose: str = """
  773. To find how many numbers in my list are less than or equal to
  774. the <number> you specify.
  775. """ # description used to instruct the LLM on when/how to use the tool
  776. number: int # required argument to the tool
  777. ```
  778. Then define a `SpyGameAgent` as a subclass of `ChatAgent`,
  779. with a method `probe` that handles this tool:
  780. ```python
  781. class SpyGameAgent(lr.ChatAgent):
  782. def __init__(self, config: lr.ChatAgentConfig):
  783. super().__init__(config)
  784. self.numbers = [3, 4, 8, 11, 15, 25, 40, 80, 90]
  785. def probe(self, msg: ProbeTool) -> str:
  786. # return how many numbers in self.numbers are less or equal to msg.number
  787. return str(len([n for n in self.numbers if n <= msg.number]))
  788. ```
  789. We then instantiate the agent and enable it to use and respond to the tool:
  790. ```python
  791. spy_game_agent = SpyGameAgent(
  792. lr.ChatAgentConfig(
  793. name="Spy",
  794. vecdb=None,
  795. use_tools=False, # don't use Langroid native tool
  796. use_functions_api=True, # use OpenAI function-call API
  797. )
  798. )
  799. spy_game_agent.enable_message(ProbeTool)
  800. ```
  801. For a full working example see the
  802. [chat-agent-tool.py](https://github.com/langroid/langroid-examples/blob/main/examples/quick-start/chat-agent-tool.py)
  803. script in the `langroid-examples` repo.
  804. </details>
  805. <details>
  806. <summary> <b>Tool/Function-calling to extract structured information from text </b> </summary>
  807. Suppose you want an agent to extract
  808. the key terms of a lease, from a lease document, as a nested JSON structure.
  809. First define the desired structure via Pydantic models:
  810. ```python
  811. from pydantic import BaseModel
  812. class LeasePeriod(BaseModel):
  813. start_date: str
  814. end_date: str
  815. class LeaseFinancials(BaseModel):
  816. monthly_rent: str
  817. deposit: str
  818. class Lease(BaseModel):
  819. period: LeasePeriod
  820. financials: LeaseFinancials
  821. address: str
  822. ```
  823. Then define the `LeaseMessage` tool as a subclass of Langroid's `ToolMessage`.
  824. Note the tool has a required argument `terms` of type `Lease`:
  825. ```python
  826. import langroid as lr
  827. class LeaseMessage(lr.agent.ToolMessage):
  828. request: str = "lease_info"
  829. purpose: str = """
  830. Collect information about a Commercial Lease.
  831. """
  832. terms: Lease
  833. ```
  834. Then define a `LeaseExtractorAgent` with a method `lease_info` that handles this tool,
  835. instantiate the agent, and enable it to use and respond to this tool:
  836. ```python
  837. class LeaseExtractorAgent(lr.ChatAgent):
  838. def lease_info(self, message: LeaseMessage) -> str:
  839. print(
  840. f"""
  841. DONE! Successfully extracted Lease Info:
  842. {message.terms}
  843. """
  844. )
  845. return json.dumps(message.terms.dict())
  846. lease_extractor_agent = LeaseExtractorAgent()
  847. lease_extractor_agent.enable_message(LeaseMessage)
  848. ```
  849. See the [`chat_multi_extract.py`](https://github.com/langroid/langroid-examples/blob/main/examples/docqa/chat_multi_extract.py)
  850. script in the `langroid-examples` repo for a full working example.
  851. </details>
  852. <details>
  853. <summary><b> Chat with documents (file paths, URLs, etc) </b></summary>
  854. Langroid provides a specialized agent class `DocChatAgent` for this purpose.
  855. It incorporates document sharding, embedding, storage in a vector-DB,
  856. and retrieval-augmented query-answer generation.
  857. Using this class to chat with a collection of documents is easy.
  858. First create a `DocChatAgentConfig` instance, with a
  859. `doc_paths` field that specifies the documents to chat with.
  860. ```python
  861. import langroid as lr
  862. from langroid.agent.special import DocChatAgentConfig, DocChatAgent
  863. config = DocChatAgentConfig(
  864. doc_paths = [
  865. "https://en.wikipedia.org/wiki/Language_model",
  866. "https://en.wikipedia.org/wiki/N-gram_language_model",
  867. "/path/to/my/notes-on-language-models.txt",
  868. ],
  869. vecdb=lr.vector_store.QdrantDBConfig(),
  870. )
  871. ```
  872. Then instantiate the `DocChatAgent` (this ingests the docs into the vector-store):
  873. ```python
  874. agent = DocChatAgent(config)
  875. ```
  876. Then we can either ask the agent one-off questions,
  877. ```python
  878. agent.llm_response("What is a language model?")
  879. ```
  880. or wrap it in a `Task` and run an interactive loop with the user:
  881. ```python
  882. task = lr.Task(agent)
  883. task.run()
  884. ```
  885. See full working scripts in the
  886. [`docqa`](https://github.com/langroid/langroid-examples/tree/main/examples/docqa)
  887. folder of the `langroid-examples` repo.
  888. </details>
  889. <details>
  890. <summary><b> ๐Ÿ”ฅ Chat with tabular data (file paths, URLs, dataframes) </b></summary>
  891. Using Langroid you can set up a `TableChatAgent` with a dataset (file path, URL or dataframe),
  892. and query it. The Agent's LLM generates Pandas code to answer the query,
  893. via function-calling (or tool/plugin), and the Agent's function-handling method
  894. executes the code and returns the answer.
  895. Here is how you can do this:
  896. ```python
  897. import langroid as lr
  898. from langroid.agent.special import TableChatAgent, TableChatAgentConfig
  899. ```
  900. Set up a `TableChatAgent` for a data file, URL or dataframe
  901. (Ensure the data table has a header row; the delimiter/separator is auto-detected):
  902. ```python
  903. dataset = "https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv"
  904. # or dataset = "/path/to/my/data.csv"
  905. # or dataset = pd.read_csv("/path/to/my/data.csv")
  906. agent = TableChatAgent(
  907. config=TableChatAgentConfig(
  908. data=dataset,
  909. )
  910. )
  911. ```
  912. Set up a task, and ask one-off questions like this:
  913. ```python
  914. task = lr.Task(
  915. agent,
  916. name = "DataAssistant",
  917. default_human_response="", # to avoid waiting for user input
  918. )
  919. result = task.run(
  920. "What is the average alcohol content of wines with a quality rating above 7?",
  921. turns=2 # return after user question, LLM fun-call/tool response, Agent code-exec result
  922. )
  923. print(result.content)
  924. ```
  925. Or alternatively, set up a task and run it in an interactive loop with the user:
  926. ```python
  927. task = lr.Task(agent, name="DataAssistant")
  928. task.run()
  929. ```
  930. For a full working example see the
  931. [`table_chat.py`](https://github.com/langroid/langroid-examples/tree/main/examples/data-qa/table_chat.py)
  932. script in the `langroid-examples` repo.
  933. </details>
  934. ---
  935. # โค๏ธ Thank you to our [supporters](https://github.com/langroid/langroid/stargazers)
  936. If you like this project, please give it a star โญ and ๐Ÿ“ข spread the word in your network or social media:
  937. [![Share on Twitter](https://img.shields.io/twitter/url?style=social&url=https://github.com/langroid/langroid)](https://twitter.com/intent/tweet?text=Langroid%20is%20a%20powerful,%20elegant%20new%20framework%20to%20easily%20build%20%23LLM%20applications.%20You%20set%20up%20LLM-powered%20Agents%20with%20vector-stores,%20assign%20tasks,%20and%20have%20them%20collaboratively%20solve%20problems%20via%20message-transformations.%20https://github.com/langroid/langroid)
  938. [![Share on LinkedIn](https://img.shields.io/badge/Share%20on-LinkedIn-blue)](https://www.linkedin.com/shareArticle?mini=true&url=https://github.com/langroid/langroid&title=Langroid:%20A%20Powerful,%20Elegant%20Framework&summary=Langroid%20is%20a%20powerful,%20elegant%20new%20framework%20to%20easily%20build%20%23LLM%20applications.%20You%20set%20up%20LLM-powered%20Agents%20with%20vector-stores,%20assign%20tasks,%20and%20have%20them%20collaboratively%20solve%20problems%20via%20message-transformations.)
  939. [![Share on Hacker News](https://img.shields.io/badge/-Share%20on%20Hacker%20News-orange)](https://news.ycombinator.com/submitlink?u=https%3A%2F%2Fgithub.com%2Flangroid%2Flangroid&t=Harness%20LLMs%20with%20Multi-Agent%20Programming)
  940. [![Share on Reddit](https://img.shields.io/badge/-Share%20on%20Reddit-blue)](https://www.reddit.com/submit?url=https%3A%2F%2Fgithub.com%2Flangroid%2Flangroid&title=Harness%20LLMs%20with%20Multi-Agent%20Programming)
  941. Your support will help build Langroid's momentum and community.
  942. # Langroid Co-Founders
  943. - [Prasad Chalasani](https://www.linkedin.com/in/pchalasani/) (IIT BTech/CS, CMU PhD/ML; Independent ML Consultant)
  944. - [Somesh Jha](https://www.linkedin.com/in/somesh-jha-80208015/) (IIT BTech/CS, CMU PhD/CS; Professor of CS, U Wisc at Madison)

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If itโ€™s helped, pass it on.