Skip to content
Back to search

langsmith-skills

MarketplaceWorks with Claude CodeGitHub

> ⚠️ — This project is in early development. APIs and skill content may change.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/langchain-ai/langsmith-skills with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/langchain-ai/langsmith-skills'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/langchain-ai-langsmith-skills/badge.svg)](/listings/langchain-ai-langsmith-skills)

HTML

<a href="/listings/langchain-ai-langsmith-skills"><img src="/api/listings/langchain-ai-langsmith-skills/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Quick Install

Using [`npx skills`](https://github.com/vercel-labs/skills):

**Local** (current project):

npx skills add langchain-ai/langsmith-skills --skill '*' --yes

**Global** (all projects):

npx skills add langchain-ai/langsmith-skills --skill '*' --yes --global

To link skills to a specific agent (e.g. Claude Code):

npx skills add langchain-ai/langsmith-skills --agent claude-code --skill '*' --yes --global

---

Claude Code Plugin

Install directly as a [Claude Code plugin](https://code.claude.com/docs/en/plugins):

/plugin marketplace add langchain-ai/langsmith-skills
/plugin install langsmith-skills@langsmith-skills

---

Install Script (Claude Code & Deep Agents CLI only)

Alternatively, clone the repo and use the install script:

# Install for Claude Code in current directory (default)
./install.sh

# Install for Claude Code in a specific project directory
./install.sh ~/my-project

# Install for Claude Code globally
./install.sh --global

# Install for DeepAgents CLI in a specific project directory
./install.sh --deepagents ~/my-project

# Install for DeepAgents CLI globally (includes agent persona)
./install.sh --deepagents --global

| Flag / Argument | Description | |------|-------------| | `DIRECTORY` | Target project directory (default: current directory, ignored with `--global`) | | `--claude` | Install for Claude Code (default) | | `--deepagents` | Install for DeepAgents CLI | | `--global`, `-g` | Install globally instead of current directory | | `--force`, `-f` | Overwrite skills with same names as this package | | `--yes`, `-y` | Skip confirmation prompts |

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 18 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. {
  2. "name": "langsmith-skills",
  3. "owner": {
  4. "name": "LangChain",
  5. "email": "support@langchain.dev"
  6. },
  7. "metadata": {
  8. "description": "Agent skills for observing and evaluating LLM applications with LangSmith"
  9. },
  10. "plugins": [
  11. {
  12. "name": "langsmith-skills",
  13. "source": "./",
  14. "description": "Agent skills for observing and evaluating LLM applications with LangSmith",
  15. "homepage": "https://github.com/langchain-ai/langsmith-skills"
  16. }
  17. ]
  18. }
README.md · README · 123 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # LangSmith Skills
  2. > **⚠️** — This project is in early development. APIs and skill content may change.
  3. Agent skills for observing and evaluating LLM applications with [LangSmith](https://smith.langchain.com/). Query traces, build evaluation datasets, and create custom evaluators — all from your coding agent.
  4. > Looking for skills to **build and improve** agents with LangChain, LangGraph, or Deep Agents? See [langchain-skills](https://github.com/langchain-ai/langchain-skills).
  5. ## Supported Coding Agents
  6. These skills can be installed for any agent supported by [`skills.sh`](https://skills.sh), including Claude Code, [Deep Agents CLI](https://docs.langchain.com/oss/python/deepagents/cli/overview), Cursor, Windsurf, Goose, and [many more](https://github.com/vercel-labs/skills).
  7. ## Prerequisites
  8. - A [LangSmith API key](https://smith.langchain.com/)
  9. ## Installation
  10. ### Quick Install
  11. Using [`npx skills`](https://github.com/vercel-labs/skills):
  12. **Local** (current project):
  13. ```bash
  14. npx skills add langchain-ai/langsmith-skills --skill '*' --yes
  15. ```
  16. **Global** (all projects):
  17. ```bash
  18. npx skills add langchain-ai/langsmith-skills --skill '*' --yes --global
  19. ```
  20. To link skills to a specific agent (e.g. Claude Code):
  21. ```bash
  22. npx skills add langchain-ai/langsmith-skills --agent claude-code --skill '*' --yes --global
  23. ```
  24. ---
  25. ### Claude Code Plugin
  26. Install directly as a [Claude Code plugin](https://code.claude.com/docs/en/plugins):
  27. ```bash
  28. /plugin marketplace add langchain-ai/langsmith-skills
  29. /plugin install langsmith-skills@langsmith-skills
  30. ```
  31. ---
  32. ### Install Script (Claude Code & Deep Agents CLI only)
  33. Alternatively, clone the repo and use the install script:
  34. ```bash
  35. # Install for Claude Code in current directory (default)
  36. ./install.sh
  37. # Install for Claude Code in a specific project directory
  38. ./install.sh ~/my-project
  39. # Install for Claude Code globally
  40. ./install.sh --global
  41. # Install for DeepAgents CLI in a specific project directory
  42. ./install.sh --deepagents ~/my-project
  43. # Install for DeepAgents CLI globally (includes agent persona)
  44. ./install.sh --deepagents --global
  45. ```
  46. | Flag / Argument | Description |
  47. |------|-------------|
  48. | `DIRECTORY` | Target project directory (default: current directory, ignored with `--global`) |
  49. | `--claude` | Install for Claude Code (default) |
  50. | `--deepagents` | Install for DeepAgents CLI |
  51. | `--global`, `-g` | Install globally instead of current directory |
  52. | `--force`, `-f` | Overwrite skills with same names as this package |
  53. | `--yes`, `-y` | Skip confirmation prompts |
  54. ## Usage
  55. After installation, set your API keys:
  56. ```bash
  57. export LANGSMITH_API_KEY=<your-key>
  58. export OPENAI_API_KEY=<your-key> # For OpenAI models
  59. export ANTHROPIC_API_KEY=<your-key> # For Anthropic models
  60. ```
  61. Then run your coding agent from the directory where you installed (for local installs) or from anywhere (for global installs).
  62. ## Available Skills (7)
  63. ### LangSmith
  64. - **langsmith-trace** - Query and export traces (includes helper scripts)
  65. - **langsmith-dataset** - Generate evaluation datasets from traces (includes helper scripts)
  66. - **langsmith-evaluator** - Create custom evaluators (includes helper scripts)
  67. - **langsmith-graph-evaluator** - Build evaluators as a graph of code, LLM and decision-model steps with `openevals.graph` (experimental)
  68. - **langsmith-experiment-diagnosis** - Find low-scoring experiment examples, explain likely causes, and propose fixes
  69. - **langsmith-custom-apps** - Build, replicate, verify, and share LangSmith Custom Apps
  70. - **langsmith-online-eval-engineering** - Design, test, and attach online evaluators for production traces and conversation threads
  71. **Note:** Several skills include Python and TypeScript helper scripts for common operations.
  72. ## Development
  73. Agent configuration lives in `config/`. To update an existing installation:
  74. ```bash
  75. ./install.sh --force
  76. ```
  77. ### Checking skill CLI commands
  78. Skills should only reference commands that exist in the released CLI. `scripts/check_cli_commands.py` extracts every `langsmith ...` command from `config/` (shell code blocks and inline code) and checks the command path and each flag against `langsmith <command> --help`. Unknown commands, singular aliases such as `langsmith trace` (use `traces`), and flags that `--help` does not list, including hidden ones, fail the check. `--api-key` is the one allowed hidden flag. Positional arguments are not checked.
  79. ```bash
  80. uv run --no-project scripts/check_cli_commands.py # all of config/
  81. uv run --no-project scripts/check_cli_commands.py path/to/SKILL.md
  82. ```
  83. It uses the `langsmith` on your `PATH` (override with `LANGSMITH_BIN`). The `skills-cli-check` workflow runs it against the latest CLI release on every pull request and weekly.
install.sh · Bundled script · 241 lines · 1 finding marked
  1. #!/bin/bash
  2. # Install LangGraph + LangSmith skills for Claude Code or Deep Agents CLI
  3. set -e
  4. SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
  5. # Default values
  6. TARGET="claude" # claude or deepagents
  7. GLOBAL=false
  8. FORCE=false
  9. YES=false
  10. LANGSMITH_ONLY=false
  11. TARGET_DIR=""
  12. # Usage
  13. usage() {
  14. echo "Usage: $0 [OPTIONS] [DIRECTORY]"
  15. echo ""
  16. echo "Install LangGraph + LangSmith skills for Claude Code or DeepAgents CLI."
  17. echo ""
  18. echo "Arguments:"
  19. echo " DIRECTORY Target project directory (default: current directory)"
  20. echo " Ignored when --global is used"
  21. echo ""
  22. echo "Options:"
  23. echo " --claude Install for Claude Code (default)"
  24. echo " --deepagents Install for Deep Agents CLI"
  25. echo " --global, -g Install globally (~/.claude or ~/.deepagents/langsmith_agent)"
  26. echo " Default: install in current directory"
  27. echo " --force, -f Overwrite skills with same names as this package"
  28. echo " --yes, -y Skip confirmation prompts"
  29. echo " --langsmith Install only LangSmith skills"
  30. echo " --help, -h Show this help message"
  31. echo ""
  32. echo "Examples:"
  33. echo " $0 # Install for Claude Code in current directory"
  34. echo " $0 ~/my-project # Install for Claude Code in ~/my-project"
  35. echo " $0 --global # Install for Claude Code globally"
  36. echo " $0 --deepagents ~/my-project # Install for Deep Agents in ~/my-project"
  37. echo " $0 -f -y # Force reinstall without prompts"
  38. exit 0
  39. }
  40. # Parse arguments
  41. while [[ $# -gt 0 ]]; do
  42. case $1 in
  43. --claude)
  44. TARGET="claude"
  45. shift
  46. ;;
  47. --deepagents)
  48. TARGET="deepagents"
  49. shift
  50. ;;
  51. --global|-g)
  52. GLOBAL=true
  53. shift
  54. ;;
  55. --force|-f)
  56. FORCE=true
  57. shift
  58. ;;
  59. --yes|-y)
  60. YES=true
  61. shift
  62. ;;
  63. --langsmith)
  64. LANGSMITH_ONLY=true
  65. shift
  66. ;;
  67. --help|-h)
  68. usage
  69. ;;
  70. -*)
  71. echo "Unknown option: $1"
  72. usage
  73. ;;
  74. *)
  75. if [ -n "$TARGET_DIR" ]; then
  76. echo "Error: multiple directories specified: '$TARGET_DIR' and '$1'"
  77. exit 1
  78. fi
  79. TARGET_DIR="$1"
  80. shift
  81. ;;
  82. esac
  83. done
  84. # Resolve target directory (default to current directory)
  85. if [ -z "$TARGET_DIR" ]; then
  86. TARGET_DIR="$(pwd)"
  87. fi
  88. # Determine installation directory and whether to include AGENTS.md
  89. INCLUDE_AGENTS_MD=false
  90. if [ "$TARGET" = "claude" ]; then
  91. if [ "$GLOBAL" = true ]; then
  92. INSTALL_DIR="$HOME/.claude"
  93. else
  94. INSTALL_DIR="$TARGET_DIR/.claude"
  95. fi
  96. TOOL_NAME="Claude Code"
  97. else
  98. # Deep Agents
  99. if [ "$GLOBAL" = true ]; then
  100. # Global: install as agent with persona
  101. INSTALL_DIR="$HOME/.deepagents/langsmith_agent"
  102. INCLUDE_AGENTS_MD=true
  103. else
  104. # Local: just skills, no agent persona
  105. INSTALL_DIR="$TARGET_DIR/.deepagents"
  106. fi
  107. TOOL_NAME="Deep Agents CLI"
  108. fi
  109. echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
  110. echo "LangChain Skills Installer"
  111. echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
  112. echo ""
  113. echo "Target: $TOOL_NAME"
  114. echo "Location: $INSTALL_DIR"
  115. if [ "$GLOBAL" = true ]; then
  116. echo "Scope: Global (all projects)"
  117. else
  118. echo "Scope: Local (current directory)"
  119. fi
  120. if [ "$LANGSMITH_ONLY" = true ]; then
  121. echo "Filter: Only LangSmith skills"
  122. fi
  123. echo ""
  124. # For Deep Agents global, check if agent already exists
  125. if [ "$TARGET" = "deepagents" ] && [ "$GLOBAL" = true ] && [ -d "$INSTALL_DIR" ]; then
  126. if [ "$FORCE" = true ]; then
  127. echo "⚠️ Existing agent found. Will overwrite (--force)."
  128. else
  129. echo "❌ ERROR: Agent 'langsmith_agent' already exists at $INSTALL_DIR"
  130. echo ""
  131. echo "To reinstall, use --force flag:"
  132. echo " $0 --deepagents --global --force"
  133. echo ""
  134. echo "Or manually remove:"
  135. echo " rm -rf $INSTALL_DIR"
  136. exit 1
  137. fi
  138. fi
  139. # Confirm installation
  140. if [ "$YES" != true ]; then
  141. read -p "Proceed with installation? (y/n): " -n 1 -r
  142. echo ""
  143. if [[ ! $REPLY =~ ^[Yy]$ ]]; then
  144. echo "Installation cancelled."
  145. exit 0
  146. fi
  147. fi
  148. echo ""
  149. echo "Installing..."
  150. # Install LangSmith CLI (best-effort, don't block on failure)
  151. if ! command -v langsmith &>/dev/null; then
  152. echo "Installing LangSmith CLI..."
  153. if curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh 2>/dev/null; then
    • Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
    • Line 167 on GitHub
  154. echo "✓ LangSmith CLI installed"
  155. else
  156. echo "⚠️ LangSmith CLI installation failed (non-critical, continuing...)"
  157. fi
  158. fi
  159. # For Deep Agents global with --force, remove existing agent
  160. if [ "$TARGET" = "deepagents" ] && [ "$GLOBAL" = true ] && [ "$FORCE" = true ] && [ -d "$INSTALL_DIR" ]; then
  161. rm -rf "$INSTALL_DIR"
  162. fi
  163. # Create directory structure
  164. mkdir -p "$INSTALL_DIR"
  165. # Copy AGENTS.md only for global Deep Agents install
  166. if [ "$INCLUDE_AGENTS_MD" = true ]; then
  167. if [ -f "$SCRIPT_DIR/config/AGENTS.md" ]; then
  168. cp "$SCRIPT_DIR/config/AGENTS.md" "$INSTALL_DIR/AGENTS.md"
  169. echo "✓ Copied AGENTS.md (agent persona)"
  170. else
  171. echo "❌ ERROR: config/AGENTS.md not found"
  172. exit 1
  173. fi
  174. fi
  175. # Copy skills
  176. if [ -d "$SCRIPT_DIR/config/skills" ]; then
  177. mkdir -p "$INSTALL_DIR/skills"
  178. for skill in "$SCRIPT_DIR/config/skills"/*; do
  179. skill_name=$(basename "$skill")
  180. # If --langsmith is provided, only install skills prefixed with "langsmith-"
  181. if [ "$LANGSMITH_ONLY" = true ] && [[ ! "$skill_name" == langsmith-* ]]; then
  182. continue
  183. fi
  184. if [ -d "$INSTALL_DIR/skills/$skill_name" ]; then
  185. if [ "$FORCE" = true ]; then
  186. rm -rf "$INSTALL_DIR/skills/$skill_name"
  187. else
  188. echo "⚠️ Skipping $skill_name (already exists, use --force to overwrite)"
  189. continue
  190. fi
  191. fi
  192. cp -r "$skill" "$INSTALL_DIR/skills/$skill_name"
  193. echo "✓ Installed $skill_name"
  194. done
  195. else
  196. echo "❌ ERROR: config/skills directory not found"
  197. exit 1
  198. fi
  199. echo ""
  200. echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
  201. echo "✅ Installation complete!"
  202. echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
  203. echo ""
  204. echo "Skills installed to $TOOL_NAME!"
  205. echo ""
  206. echo "To see installed skills:"
  207. echo " ls $INSTALL_DIR/skills/"
  208. echo ""
  209. if [ "$TARGET" = "deepagents" ]; then
  210. if [ "$GLOBAL" = true ]; then
  211. echo "To use this agent, run:"
  212. echo " deepagents --agent langsmith_agent"
  213. echo ""
  214. fi
  215. echo "For usage, configuration, and API key setup, see the CLI docs:"
  216. echo " https://docs.langchain.com/deepagents-cli"
  217. else
  218. echo "For usage, configuration, and API key setup, see the Claude Code docs:"
  219. echo " https://code.claude.com/docs/en/overview"
  220. fi
  221. echo ""

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.