Skip to content
Back to search

langsmith-mcp-server

A production-ready Model Context Protocol (MCP) server that provides seamless integration with the LangSmith observability platform. This server enables language models to fetch conversation history,…

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/langchain-ai--langsmith-mcp-server/badge.svg)](/listings/langchain-ai--langsmith-mcp-server)

HTML

<a href="/listings/langchain-ai--langsmith-mcp-server"><img src="/api/listings/langchain-ai--langsmith-mcp-server/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

A **hosted version** of the LangSmith MCP Server is available over HTTP-streamable transport, so you can connect without running the server yourself:

- **URL:** `https://langsmith-mcp-server.onrender.com/mcp` - **Hosting:** [Render](https://render.com), built from this public repo using the project's Dockerfile.

Use it like any HTTP-streamable MCP server: point your client at the URL and send your LangSmith API key in the `LANGSMITH-API-KEY` header. No local install or Docker required.

**Example (Cursor `mcp.json`):**

{
  "mcpServers": {
    "LangSmith MCP (Hosted)": {
      "url": "https://langsmith-mcp-server.onrender.com/mcp",
      "headers": {
        "LANGSMITH-API-KEY": "lsv2_pt_your_api_key_here"
      }
    }
  }
}

Optional headers: `LANGSMITH-WORKSPACE-ID`, `LANGSMITH-ENDPOINT` (same as in the [Docker Deployment](#-docker-deployment-http-streamable) section below).

> **Note:** This deployed instance is intended for [LangSmith Cloud](https://smith.langchain.com). If you use a **self-hosted** LangSmith instance, run the server yourself and point it at your endpoint—see the [Docker Deployment](#-docker-deployment-http-streamable) section below.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 448 lines · 1 finding marked

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # 🦜🛠️ LangSmith MCP Server
  2. ![LangSmith MCP Hero](https://raw.githubusercontent.com/langchain-ai/langsmith-mcp-server/refs/heads/main/docs/assets/langsmith_mcp_hero.png)
  3. [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
  4. [![Python 3.10](https://img.shields.io/badge/python-3.10-blue.svg)](https://www.python.org/downloads/release/python-3100/)
  5. A production-ready [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) server that provides seamless integration with the [LangSmith](https://smith.langchain.com) observability platform. This server enables language models to fetch conversation history, prompts, runs and traces, datasets, experiments, and billing usage from LangSmith.
  6. ## 📋 Example Use Cases
  7. The server enables powerful capabilities including:
  8. - 💬 **Conversation History**: "Fetch the history of my conversation from thread 'thread-123' in project 'my-chatbot'" (paginated by character budget)
  9. - 📚 **Prompt Management**: "Get all public prompts in my workspace" / "Pull the template for the 'legal-case-summarizer' prompt"
  10. - 🔍 **Traces & Runs**: "Fetch the latest 10 root runs from project 'alpha'" / "Get all runs for trace &lt;uuid&gt; (page 2 of 5)"
  11. - 📊 **Datasets**: "List datasets of type chat" / "Read examples from dataset 'customer-support-qa'"
  12. - 🧪 **Experiments**: "List experiments for dataset 'my-eval-set' with latency and cost metrics"
  13. - 📈 **Billing**: "Get billing usage for September 2025"
  14. ## 🚀 Quickstart
  15. A **hosted version** of the LangSmith MCP Server is available over HTTP-streamable transport, so you can connect without running the server yourself:
  16. - **URL:** `https://langsmith-mcp-server.onrender.com/mcp`
  17. - **Hosting:** [Render](https://render.com), built from this public repo using the project's Dockerfile.
  18. Use it like any HTTP-streamable MCP server: point your client at the URL and send your LangSmith API key in the `LANGSMITH-API-KEY` header. No local install or Docker required.
  19. **Example (Cursor `mcp.json`):**
  20. ```json
  21. {
  22. "mcpServers": {
  23. "LangSmith MCP (Hosted)": {
  24. "url": "https://langsmith-mcp-server.onrender.com/mcp",
  25. "headers": {
  26. "LANGSMITH-API-KEY": "lsv2_pt_your_api_key_here"
  27. }
  28. }
  29. }
  30. }
  31. ```
  32. Optional headers: `LANGSMITH-WORKSPACE-ID`, `LANGSMITH-ENDPOINT` (same as in the [Docker Deployment](#-docker-deployment-http-streamable) section below).
  33. > **Note:** This deployed instance is intended for [LangSmith Cloud](https://smith.langchain.com). If you use a **self-hosted** LangSmith instance, run the server yourself and point it at your endpoint—see the [Docker Deployment](#-docker-deployment-http-streamable) section below.
  34. ## 🛠️ Available Tools
  35. The LangSmith MCP Server provides the following tools for integration with LangSmith.
  36. ### 💬 Conversation & Threads
  37. | Tool Name | Description |
  38. |-----------|-------------|
  39. | `get_thread_history` | Retrieve message history for a conversation thread. Uses **char-based pagination**: pass `page_number` (1-based), and use returned `total_pages` to request more pages. Optional `max_chars_per_page` and `preview_chars` control page size and long-string truncation. |
  40. ### 📚 Prompt Management
  41. | Tool Name | Description |
  42. |-----------|-------------|
  43. | `list_prompts` | Fetch prompts from LangSmith with optional filtering by visibility (public/private) and limit. |
  44. | `get_prompt_by_name` | Get a specific prompt by its exact name, returning the prompt details and template. |
  45. | `push_prompt` | Documentation-only: how to create and push prompts to LangSmith. |
  46. ### 🔍 Traces & Runs
  47. | Tool Name | Description |
  48. |-----------|-------------|
  49. | `fetch_runs` | Fetch LangSmith runs (traces, tools, chains, etc.) from one or more projects. Supports filters (run_type, error, is_root), FQL (`filter`, `trace_filter`, `tree_filter`), and ordering. When `trace_id` is set, returns **char-based paginated** pages; otherwise returns one batch up to `limit`. Always pass `limit` and `page_number`. |
  50. | `list_projects` | List LangSmith projects with optional filtering by name, dataset, and detail level (simplified vs full). |
  51. ### 📊 Datasets & Examples
  52. | Tool Name | Description |
  53. |-----------|-------------|
  54. | `list_datasets` | Fetch datasets with filtering by ID, type, name, name substring, or metadata. |
  55. | `list_examples` | Fetch examples from a dataset by dataset ID/name or example IDs, with filter, metadata, splits, and optional `as_of` version. |
  56. | `read_dataset` | Read a single dataset by ID or name. |
  57. | `read_example` | Read a single example by ID, with optional `as_of` version. |
  58. | `create_dataset` | Documentation-only: how to create datasets in LangSmith. |
  59. | `update_examples` | Documentation-only: how to update dataset examples in LangSmith. |
  60. ### 🧪 Experiments & Evaluations
  61. | Tool Name | Description |
  62. |-----------|-------------|
  63. | `list_experiments` | List experiment projects (reference projects) for a dataset. Requires `reference_dataset_id` or `reference_dataset_name`. Returns key metrics (latency, cost, feedback stats). |
  64. | `run_experiment` | Documentation-only: how to run experiments and evaluations in LangSmith. |
  65. ### 📈 Usage & Billing
  66. | Tool Name | Description |
  67. |-----------|-------------|
  68. | `get_billing_usage` | Fetch organization billing usage (e.g. trace counts) for a date range. Optional workspace filter; returns metrics with workspace names inline. |
  69. ### 📄 Pagination (char-based)
  70. Several tools use **stateless, character-budget pagination** so responses stay within a size limit and work well with LLM clients:
  71. - **Where it’s used:** `get_thread_history` and `fetch_runs` (when `trace_id` is set).
  72. - **Parameters:** You send `page_number` (1-based) on every request. Optional: `max_chars_per_page` (default 25000, cap 30000) and `preview_chars` (truncate long strings with "… (+N chars)").
  73. - **Response:** Each response includes `page_number`, `total_pages`, and the page payload (`result` for messages, `runs` for runs). To get more, call again with `page_number = 2`, then `3`, up to `total_pages`.
  74. - **Why it’s useful:** Pages are built by JSON character count, not item count, so each page fits within a fixed size. No cursor or server-side state—just integer page numbers.
  75. ## 🛠️ Installation Options
  76. ### 📝 General Prerequisites
  77. 1. Install [uv](https://github.com/astral-sh/uv) (a fast Python package installer and resolver):
  78. ```bash
  79. curl -LsSf https://astral.sh/uv/install.sh | sh
  80. ```
  81. 2. Clone this repository and navigate to the project directory:
  82. ```bash
  83. git clone https://github.com/langchain-ai/langsmith-mcp-server.git
  84. cd langsmith-mcp-server
  85. ```
  86. ### 🔌 MCP Client Integration
  87. Once you have the LangSmith MCP Server, you can integrate it with various MCP-compatible clients. You have two installation options:
  88. #### 📦 From PyPI
  89. 1. Install the package:
  90. ```bash
  91. uv run pip install --upgrade langsmith-mcp-server
  92. ```
  93. 2. Add to your client MCP config:
  94. ```json
  95. {
  96. "mcpServers": {
  97. "LangSmith API MCP Server": {
  98. "command": "/path/to/uvx",
  99. "args": [
  100. "langsmith-mcp-server"
  101. ],
  102. "env": {
  103. "LANGSMITH_API_KEY": "your_langsmith_api_key",
  104. "LANGSMITH_WORKSPACE_ID": "your_workspace_id",
  105. "LANGSMITH_ENDPOINT": "https://api.smith.langchain.com"
  106. }
  107. }
  108. }
  109. }
  110. ```
  111. #### ⚙️ From Source
  112. Add the following configuration to your MCP client settings (run from the **project root** so the package is found):
  113. ```json
  114. {
  115. "mcpServers": {
  116. "LangSmith API MCP Server": {
  117. "command": "/path/to/uv",
  118. "args": [
  119. "--directory",
  120. "/path/to/langsmith-mcp-server",
  121. "run",
  122. "langsmith_mcp_server/server.py"
  123. ],
  124. "env": {
  125. "LANGSMITH_API_KEY": "your_langsmith_api_key",
  126. "LANGSMITH_WORKSPACE_ID": "your_workspace_id",
  127. "LANGSMITH_ENDPOINT": "https://api.smith.langchain.com"
  128. }
  129. }
  130. }
  131. }
  132. ```
  133. Replace the following placeholders:
  134. - `/path/to/uv`: The absolute path to your uv installation (e.g., `/Users/username/.local/bin/uv`). You can find it with `which uv`.
  135. - `/path/to/langsmith-mcp-server`: The absolute path to the **project root** (the directory containing `pyproject.toml` and `langsmith_mcp_server/`).
  136. - `your_langsmith_api_key`: Your LangSmith API key (required).
  137. - `your_workspace_id`: Your LangSmith workspace ID (optional, for API keys scoped to multiple workspaces).
  138. - `https://api.smith.langchain.com`: The LangSmith API endpoint (optional, defaults to the standard endpoint).
  139. Example configuration (PyPI/uvx):
  140. ```json
  141. {
  142. "mcpServers": {
  143. "LangSmith API MCP Server": {
  144. "command": "/path/to/uvx",
  145. "args": ["langsmith-mcp-server"],
  146. "env": {
  147. "LANGSMITH_API_KEY": "lsv2_pt_your_key_here",
  148. "LANGSMITH_WORKSPACE_ID": "your_workspace_id",
  149. "LANGSMITH_ENDPOINT": "https://api.smith.langchain.com"
  150. }
  151. }
  152. }
  153. }
  154. ```
  155. Copy this configuration into Cursor → MCP Settings (replace `/path/to/uvx` with the output of `which uvx`).
  156. ![LangSmith Cursor Integration](docs/assets/cursor_mcp.png)
  157. ### 🔧 Headers (tool invocation)
  158. When connecting over **HTTP** (e.g. streamable HTTP or a hosted MCP endpoint), the server uses **headers** for authentication and configuration. Your MCP client must send these with each request; no environment variables are required for tool invocation.
  159. | Header | Required | Description |
  160. |--------|----------|-------------|
  161. | `LANGSMITH-API-KEY` | ✅ Yes | Your LangSmith API key for tool calls (list prompts, fetch runs, etc.) |
  162. | `LANGSMITH-WORKSPACE-ID` | ❌ No | Workspace ID for API keys scoped to multiple workspaces |
  163. | `LANGSMITH-ENDPOINT` | ❌ No | Custom API endpoint URL (for self-hosted or EU region) |
  164. Optional headers used only when [server monitoring](#optional-tool-call-monitoring-to-a-second-langsmith-instance) is enabled (for grouping traces by session):
  165. | Header | Description |
  166. |--------|-------------|
  167. | `mcp-session-id` | Session or thread id; stored in trace metadata as `session_id` |
  168. | `x-session-id` | Fallback if `mcp-session-id` is not set |
  169. | `x-request-id` | Fallback for request-scoped grouping |
  170. **Stdio transport:** When running the server over stdio (e.g. `uvx langsmith-mcp-server`), there are no headers. The server falls back to the **environment variables** `LANGSMITH_API_KEY`, `LANGSMITH_WORKSPACE_ID`, and `LANGSMITH_ENDPOINT` in the process environment so that tool invocation still works.
  171. ---
  172. ### 🔧 Environment variables
  173. Environment variables are **not** used for tool invocation when using HTTP (headers are). They are used for:
  174. 1. **Stdio transport** – fallback for credentials when no headers exist (see above).
  175. 2. **Load tests** – e.g. `tests/load_test_sessions.py` reads `LANGSMITH_API_KEY` from the environment (or a `.env` file at the project root).
  176. 3. **Optional server monitoring** – tracing tool calls to a second LangSmith instance (see below).
  177. | Variable | Used for | Description |
  178. |----------|----------|-------------|
  179. | `LANGSMITH_API_KEY` | Stdio fallback, load tests | LangSmith API key (when not provided via headers) |
  180. | `LANGSMITH_WORKSPACE_ID` | Stdio fallback | Workspace ID (optional) |
  181. | `LANGSMITH_ENDPOINT` | Stdio fallback | Custom endpoint URL (optional) |
  182. #### Optional: Tool-call monitoring to a second LangSmith instance
  183. You can log every MCP tool call (with inputs and outputs) to a **separate** LangSmith project for monitoring and analytics. Set these in your environment (e.g. in a `.env` file at the project root; the server loads `.env` via `python-dotenv`):
  184. | Variable | Required | Description |
  185. |----------|----------|-------------|
  186. | `LANGSMITH_MONITORING_API_KEY` | Yes (to enable) | API key for the LangSmith instance used for monitoring |
  187. | `LANGSMITH_MONITORING_ENDPOINT` | No | Endpoint URL (default: cloud) |
  188. | `LANGSMITH_MONITORING_WORKSPACE_ID` | No | Workspace ID for the monitoring instance |
  189. | `LANGSMITH_MONITORING_PROJECT` | No | Project name for monitoring traces (default: `mcp-server-monitoring`) |
  190. | `LANGSMITH_TRACING` | Yes (to send traces) | Set to `true` so traces are sent to LangSmith ([custom instrumentation](https://docs.smith.langchain.com)) |
  191. Each tool run is traced with `run_type="tool"` and a **session_id** in metadata (from the `mcp-session-id`, `x-session-id`, or `x-request-id` header when using HTTP, or generated per request).
  192. If you use the **hosted** LangSmith MCP Server, anonymous usage data is sent to a separate LangSmith project so we can iterate and improve the product.
  193. ## 🐳 Docker Deployment (HTTP-Streamable)
  194. The LangSmith MCP Server can be deployed as an HTTP server using Docker, enabling remote access via the HTTP-streamable protocol.
  195. ### Building the Docker Image
  196. ```bash
  197. docker build -t langsmith-mcp-server .
  198. ```
  199. ### Running with Docker
  200. ```bash
  201. docker run -p 8000:8000 langsmith-mcp-server
  202. ```
  203. The API key is provided via the `LANGSMITH-API-KEY` header when connecting, so no environment variables are required for HTTP-streamable protocol.
  204. ### Connecting with HTTP-Streamable Protocol
  205. Once the Docker container is running, you can connect to it using the HTTP-streamable transport. The server accepts authentication via headers:
  206. **Required header:**
  207. - `LANGSMITH-API-KEY`: Your LangSmith API key
  208. **Optional headers:**
  209. - `LANGSMITH-WORKSPACE-ID`: Workspace ID for API keys scoped to multiple workspaces
  210. - `LANGSMITH-ENDPOINT`: Custom endpoint URL (for self-hosted or EU region)
  211. **Example client configuration:**
  212. ```python
  213. from mcp import ClientSession
  214. from mcp.client.streamable_http import streamablehttp_client
  215. headers = {
  216. "LANGSMITH-API-KEY": "lsv2_pt_your_api_key_here",
  217. # Optional:
  218. # "LANGSMITH-WORKSPACE-ID": "your_workspace_id",
  219. # "LANGSMITH-ENDPOINT": "https://api.smith.langchain.com",
  220. }
  221. async with streamablehttp_client("http://localhost:8000/mcp", headers=headers) as (read, write, _):
  222. async with ClientSession(read, write) as session:
  223. await session.initialize()
  224. # Use the session to call tools, list prompts, etc.
  225. ```
  226. ### Cursor Integration
  227. To add the LangSmith MCP Server to Cursor using HTTP-streamable protocol, add the following to your `mcp.json` configuration file:
  228. ```json
  229. {
  230. "mcpServers": {
  231. "HTTP-Streamable LangSmith MCP Server": {
  232. "url": "http://localhost:8000/mcp",
  233. "headers": {
  234. "LANGSMITH-API-KEY": "lsv2_pt_your_api_key_here"
  235. }
  236. }
  237. }
  238. }
  239. ```
  240. **Optional headers:**
  241. ```json
  242. {
  243. "mcpServers": {
  244. "HTTP-Streamable LangSmith MCP Server": {
  245. "url": "http://localhost:8000/mcp",
  246. "headers": {
  247. "LANGSMITH-API-KEY": "lsv2_pt_your_api_key_here",
  248. "LANGSMITH-WORKSPACE-ID": "your_workspace_id",
  249. "LANGSMITH-ENDPOINT": "https://api.smith.langchain.com"
  250. }
  251. }
  252. }
  253. }
  254. ```
  255. Make sure the server is running before connecting Cursor to it.
  256. ### Health Check
  257. The server provides a health check endpoint:
  258. ```bash
  259. curl http://localhost:8000/health
  260. ```
  261. This endpoint does not require authentication and returns `"LangSmith MCP server is running"` when the server is healthy.
  262. ## 🧪 Development and Contributing
  263. ### Prerequisites
  264. - **Python** 3.10+ (3.11+ recommended)
  265. - **[uv](https://docs.astral.sh/uv/)** – install with `curl -LsSf https://astral.sh/uv/install.sh | sh`
    • Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
    • Line 354 on GitHub
  266. - **LangSmith API key** – from [smith.langchain.com](https://smith.langchain.com)
  267. - **Node.js** (optional) – only if you want to use [MCP Inspector](https://github.com/modelcontextprotocol/inspector) to test the server (stdio or streamable-http)
  268. ### Setup
  269. ```bash
  270. git clone https://github.com/langchain-ai/langsmith-mcp-server.git
  271. cd langsmith-mcp-server
  272. uv sync # Install dependencies
  273. uv sync --group test # Include test dependencies (pytest, ruff, mypy)
  274. uvx langsmith-mcp-server # Verify CLI runs (stdio)
  275. ```
  276. ### Development workflow
  277. 1. **Edit code** in `langsmith_mcp_server/` or `tests/`.
  278. 2. **Format and lint** (required before committing):
  279. ```bash
  280. make format
  281. make lint
  282. ```
  283. 3. **Run tests**:
  284. ```bash
  285. make test
  286. # Or a single file:
  287. make test TEST_FILE=tests/tools/test_dataset_tools.py
  288. ```
  289. 4. **Type-check** (optional): `uv run mypy langsmith_mcp_server/`
  290. ### Testing with MCP Inspector
  291. You can test the server with [MCP Inspector](https://github.com/modelcontextprotocol/inspector) using either **stdio** or **streamable-http**.
  292. 1. **Start MCP Inspector:**
  293. ```bash
  294. npx @modelcontextprotocol/inspector@latest
  295. ```
  296. Open **http://localhost:6274** in your browser.
  297. 2. **Connect** in the Inspector:
  298. - **Stdio:** Choose stdio transport and configure the server command (e.g. `uv run langsmith-mcp-server`) and set `LANGSMITH_API_KEY` in the environment.
  299. - **Streamable HTTP:** Start the server first (`uv run uvicorn langsmith_mcp_server.server:app --host 0.0.0.0 --port 8000` or Docker), then choose streamable-http, URL **`http://localhost:8000/mcp`**, and add header **`LANGSMITH-API-KEY`** = your API key.
  300. ### Load testing
  301. A **session-based** load test opens many MCP sessions and calls the `list_prompts` tool in each, using [langchain-mcp-adapters](https://github.com/langchain-ai/langchain-mcp-adapters). Run from the CLI (no UI). The server must be running first.
  302. ```bash
  303. uv sync --group load
  304. # Terminal 1: start the server
  305. uv run uvicorn langsmith_mcp_server.server:app --host 0.0.0.0 --port 8000
  306. # Terminal 2: run the load test
  307. uv run python tests/load_test_sessions.py --sessions 20 --calls-per-session 3
  308. ```
  309. **Options**
  310. | Option | Default | Description |
  311. |--------|---------|-------------|
  312. | `--url` | `http://localhost:8000/mcp` | MCP endpoint URL |
  313. | `--api-key` | from `.env` | `LANGSMITH_API_KEY` (or set in project root `.env`) |
  314. | `--sessions` | 10 | Number of concurrent sessions |
  315. | `--calls-per-session` | 3 | `list_prompts` calls per session |
  316. | `--debug` | off | Print step-by-step logs and first error traceback |
  317. | `--report` PATH | — | Write a report after the run (see below) |
  318. **Report**
  319. Use `--report PATH` to write a JSON report after the test (e.g. `--report load_test_report` creates `load_test_report.json` with config, summary, per-session results, and first error).
  320. ```bash
  321. uv run python tests/load_test_sessions.py --sessions 5 --report load_test_report
  322. # Creates: load_test_report.json (in current directory)
  323. ```
  324. ### Contributing checklist
  325. Before opening a PR:
  326. - [ ] `make format` and `make lint` pass
  327. - [ ] `make test` passes
  328. - [ ] New tools or behavior are documented (e.g. in [CLAUDE.md](CLAUDE.md) if you change architecture or tools)
  329. - [ ] Error handling in tools returns `{"error": "..."}` rather than raising
  330. For more detail (adding tools, code standards, troubleshooting), see **[CLAUDE.md](CLAUDE.md)**.
  331. ## 📄 License
  332. This project is distributed under the MIT License. For detailed terms and conditions, please refer to the LICENSE file.
  333. Made with ❤️ by the [LangChain](https://langchain.com) Team

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.