langsmith-skills
> ⚠️ — This project is in early development. APIs and skill content may change.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Check it deeper on your machine
It has changed since its scan on 11 Nov 2026.
As a Claude Code plugin, install it and then ask Claude Code:
/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/langchain-ai/langsmith-skills with scanskillsafety
Or from a terminal, clone it and run it:
git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/langchain-ai/langsmith-skills'
It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.
Badge for your README
Markdown
[](/listings/langchain-ai-langsmith-skills)HTML
<a href="/listings/langchain-ai-langsmith-skills"><img src="/api/listings/langchain-ai-langsmith-skills/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
Quick Install
Using [`npx skills`](https://github.com/vercel-labs/skills):
**Local** (current project):
npx skills add langchain-ai/langsmith-skills --skill '*' --yes
**Global** (all projects):
npx skills add langchain-ai/langsmith-skills --skill '*' --yes --global
To link skills to a specific agent (e.g. Claude Code):
npx skills add langchain-ai/langsmith-skills --agent claude-code --skill '*' --yes --global
---
Claude Code Plugin
Install directly as a [Claude Code plugin](https://code.claude.com/docs/en/plugins):
/plugin marketplace add langchain-ai/langsmith-skills /plugin install langsmith-skills@langsmith-skills
---
Install Script (Claude Code & Deep Agents CLI only)
Alternatively, clone the repo and use the install script:
# Install for Claude Code in current directory (default) ./install.sh # Install for Claude Code in a specific project directory ./install.sh ~/my-project # Install for Claude Code globally ./install.sh --global # Install for DeepAgents CLI in a specific project directory ./install.sh --deepagents ~/my-project # Install for DeepAgents CLI globally (includes agent persona) ./install.sh --deepagents --global
| Flag / Argument | Description | |------|-------------| | `DIRECTORY` | Target project directory (default: current directory, ignored with `--global`) | | `--claude` | Install for Claude Code (default) | | `--deepagents` | Install for DeepAgents CLI | | `--global`, `-g` | Install globally instead of current directory | | `--force`, `-f` | Overwrite skills with same names as this package | | `--yes`, `-y` | Skip confirmation prompts |
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
.claude-plugin/marketplace.json · Manifest · 18 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- {
- "name": "langsmith-skills",
- "owner": {
- "name": "LangChain",
- "email": "support@langchain.dev"
- },
- "metadata": {
- "description": "Agent skills for observing and evaluating LLM applications with LangSmith"
- },
- "plugins": [
- {
- "name": "langsmith-skills",
- "source": "./",
- "description": "Agent skills for observing and evaluating LLM applications with LangSmith",
- "homepage": "https://github.com/langchain-ai/langsmith-skills"
- }
- ]
- }
README.md · README · 123 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- # LangSmith Skills
- > **⚠️** — This project is in early development. APIs and skill content may change.
- Agent skills for observing and evaluating LLM applications with [LangSmith](https://smith.langchain.com/). Query traces, build evaluation datasets, and create custom evaluators — all from your coding agent.
- > Looking for skills to **build and improve** agents with LangChain, LangGraph, or Deep Agents? See [langchain-skills](https://github.com/langchain-ai/langchain-skills).
- ## Supported Coding Agents
- These skills can be installed for any agent supported by [`skills.sh`](https://skills.sh), including Claude Code, [Deep Agents CLI](https://docs.langchain.com/oss/python/deepagents/cli/overview), Cursor, Windsurf, Goose, and [many more](https://github.com/vercel-labs/skills).
- ## Prerequisites
- - A [LangSmith API key](https://smith.langchain.com/)
- ## Installation
- ### Quick Install
- Using [`npx skills`](https://github.com/vercel-labs/skills):
- **Local** (current project):
- ```bash
- npx skills add langchain-ai/langsmith-skills --skill '*' --yes
- ```
- **Global** (all projects):
- ```bash
- npx skills add langchain-ai/langsmith-skills --skill '*' --yes --global
- ```
- To link skills to a specific agent (e.g. Claude Code):
- ```bash
- npx skills add langchain-ai/langsmith-skills --agent claude-code --skill '*' --yes --global
- ```
- ---
- ### Claude Code Plugin
- Install directly as a [Claude Code plugin](https://code.claude.com/docs/en/plugins):
- ```bash
- /plugin marketplace add langchain-ai/langsmith-skills
- /plugin install langsmith-skills@langsmith-skills
- ```
- ---
- ### Install Script (Claude Code & Deep Agents CLI only)
- Alternatively, clone the repo and use the install script:
- ```bash
- # Install for Claude Code in current directory (default)
- ./install.sh
- # Install for Claude Code in a specific project directory
- ./install.sh ~/my-project
- # Install for Claude Code globally
- ./install.sh --global
- # Install for DeepAgents CLI in a specific project directory
- ./install.sh --deepagents ~/my-project
- # Install for DeepAgents CLI globally (includes agent persona)
- ./install.sh --deepagents --global
- ```
- | Flag / Argument | Description |
- |------|-------------|
- | `DIRECTORY` | Target project directory (default: current directory, ignored with `--global`) |
- | `--claude` | Install for Claude Code (default) |
- | `--deepagents` | Install for DeepAgents CLI |
- | `--global`, `-g` | Install globally instead of current directory |
- | `--force`, `-f` | Overwrite skills with same names as this package |
- | `--yes`, `-y` | Skip confirmation prompts |
- ## Usage
- After installation, set your API keys:
- ```bash
- export LANGSMITH_API_KEY=<your-key>
- export OPENAI_API_KEY=<your-key> # For OpenAI models
- export ANTHROPIC_API_KEY=<your-key> # For Anthropic models
- ```
- Then run your coding agent from the directory where you installed (for local installs) or from anywhere (for global installs).
- ## Available Skills (7)
- ### LangSmith
- - **langsmith-trace** - Query and export traces (includes helper scripts)
- - **langsmith-dataset** - Generate evaluation datasets from traces (includes helper scripts)
- - **langsmith-evaluator** - Create custom evaluators (includes helper scripts)
- - **langsmith-graph-evaluator** - Build evaluators as a graph of code, LLM and decision-model steps with `openevals.graph` (experimental)
- - **langsmith-experiment-diagnosis** - Find low-scoring experiment examples, explain likely causes, and propose fixes
- - **langsmith-custom-apps** - Build, replicate, verify, and share LangSmith Custom Apps
- - **langsmith-online-eval-engineering** - Design, test, and attach online evaluators for production traces and conversation threads
- **Note:** Several skills include Python and TypeScript helper scripts for common operations.
- ## Development
- Agent configuration lives in `config/`. To update an existing installation:
- ```bash
- ./install.sh --force
- ```
- ### Checking skill CLI commands
- Skills should only reference commands that exist in the released CLI. `scripts/check_cli_commands.py` extracts every `langsmith ...` command from `config/` (shell code blocks and inline code) and checks the command path and each flag against `langsmith <command> --help`. Unknown commands, singular aliases such as `langsmith trace` (use `traces`), and flags that `--help` does not list, including hidden ones, fail the check. `--api-key` is the one allowed hidden flag. Positional arguments are not checked.
- ```bash
- uv run --no-project scripts/check_cli_commands.py # all of config/
- uv run --no-project scripts/check_cli_commands.py path/to/SKILL.md
- ```
- It uses the `langsmith` on your `PATH` (override with `LANGSMITH_BIN`). The `skills-cli-check` workflow runs it against the latest CLI release on every pull request and weekly.
install.sh · Bundled script · 241 lines · 1 finding marked
- #!/bin/bash
- # Install LangGraph + LangSmith skills for Claude Code or Deep Agents CLI
- set -e
- SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
- # Default values
- TARGET="claude" # claude or deepagents
- GLOBAL=false
- FORCE=false
- YES=false
- LANGSMITH_ONLY=false
- TARGET_DIR=""
- # Usage
- usage() {
- echo "Usage: $0 [OPTIONS] [DIRECTORY]"
- echo ""
- echo "Install LangGraph + LangSmith skills for Claude Code or DeepAgents CLI."
- echo ""
- echo "Arguments:"
- echo " DIRECTORY Target project directory (default: current directory)"
- echo " Ignored when --global is used"
- echo ""
- echo "Options:"
- echo " --claude Install for Claude Code (default)"
- echo " --deepagents Install for Deep Agents CLI"
- echo " --global, -g Install globally (~/.claude or ~/.deepagents/langsmith_agent)"
- echo " Default: install in current directory"
- echo " --force, -f Overwrite skills with same names as this package"
- echo " --yes, -y Skip confirmation prompts"
- echo " --langsmith Install only LangSmith skills"
- echo " --help, -h Show this help message"
- echo ""
- echo "Examples:"
- echo " $0 # Install for Claude Code in current directory"
- echo " $0 ~/my-project # Install for Claude Code in ~/my-project"
- echo " $0 --global # Install for Claude Code globally"
- echo " $0 --deepagents ~/my-project # Install for Deep Agents in ~/my-project"
- echo " $0 -f -y # Force reinstall without prompts"
- exit 0
- }
- # Parse arguments
- while [[ $# -gt 0 ]]; do
- case $1 in
- --claude)
- TARGET="claude"
- shift
- ;;
- --deepagents)
- TARGET="deepagents"
- shift
- ;;
- --global|-g)
- GLOBAL=true
- shift
- ;;
- --force|-f)
- FORCE=true
- shift
- ;;
- --yes|-y)
- YES=true
- shift
- ;;
- --langsmith)
- LANGSMITH_ONLY=true
- shift
- ;;
- --help|-h)
- usage
- ;;
- -*)
- echo "Unknown option: $1"
- usage
- ;;
- *)
- if [ -n "$TARGET_DIR" ]; then
- echo "Error: multiple directories specified: '$TARGET_DIR' and '$1'"
- exit 1
- fi
- TARGET_DIR="$1"
- shift
- ;;
- esac
- done
- # Resolve target directory (default to current directory)
- if [ -z "$TARGET_DIR" ]; then
- TARGET_DIR="$(pwd)"
- fi
- # Determine installation directory and whether to include AGENTS.md
- INCLUDE_AGENTS_MD=false
- if [ "$TARGET" = "claude" ]; then
- if [ "$GLOBAL" = true ]; then
- INSTALL_DIR="$HOME/.claude"
- else
- INSTALL_DIR="$TARGET_DIR/.claude"
- fi
- TOOL_NAME="Claude Code"
- else
- # Deep Agents
- if [ "$GLOBAL" = true ]; then
- # Global: install as agent with persona
- INSTALL_DIR="$HOME/.deepagents/langsmith_agent"
- INCLUDE_AGENTS_MD=true
- else
- # Local: just skills, no agent persona
- INSTALL_DIR="$TARGET_DIR/.deepagents"
- fi
- TOOL_NAME="Deep Agents CLI"
- fi
- echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
- echo "LangChain Skills Installer"
- echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
- echo ""
- echo "Target: $TOOL_NAME"
- echo "Location: $INSTALL_DIR"
- if [ "$GLOBAL" = true ]; then
- echo "Scope: Global (all projects)"
- else
- echo "Scope: Local (current directory)"
- fi
- if [ "$LANGSMITH_ONLY" = true ]; then
- echo "Filter: Only LangSmith skills"
- fi
- echo ""
- # For Deep Agents global, check if agent already exists
- if [ "$TARGET" = "deepagents" ] && [ "$GLOBAL" = true ] && [ -d "$INSTALL_DIR" ]; then
- if [ "$FORCE" = true ]; then
- echo "⚠️ Existing agent found. Will overwrite (--force)."
- else
- echo "❌ ERROR: Agent 'langsmith_agent' already exists at $INSTALL_DIR"
- echo ""
- echo "To reinstall, use --force flag:"
- echo " $0 --deepagents --global --force"
- echo ""
- echo "Or manually remove:"
- echo " rm -rf $INSTALL_DIR"
- exit 1
- fi
- fi
- # Confirm installation
- if [ "$YES" != true ]; then
- read -p "Proceed with installation? (y/n): " -n 1 -r
- echo ""
- if [[ ! $REPLY =~ ^[Yy]$ ]]; then
- echo "Installation cancelled."
- exit 0
- fi
- fi
- echo ""
- echo "Installing..."
- # Install LangSmith CLI (best-effort, don't block on failure)
- if ! command -v langsmith &>/dev/null; then
- echo "Installing LangSmith CLI..."
- if curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh 2>/dev/null; then
- Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
- Line 167 on GitHub
- echo "✓ LangSmith CLI installed"
- else
- echo "⚠️ LangSmith CLI installation failed (non-critical, continuing...)"
- fi
- fi
- # For Deep Agents global with --force, remove existing agent
- if [ "$TARGET" = "deepagents" ] && [ "$GLOBAL" = true ] && [ "$FORCE" = true ] && [ -d "$INSTALL_DIR" ]; then
- rm -rf "$INSTALL_DIR"
- fi
- # Create directory structure
- mkdir -p "$INSTALL_DIR"
- # Copy AGENTS.md only for global Deep Agents install
- if [ "$INCLUDE_AGENTS_MD" = true ]; then
- if [ -f "$SCRIPT_DIR/config/AGENTS.md" ]; then
- cp "$SCRIPT_DIR/config/AGENTS.md" "$INSTALL_DIR/AGENTS.md"
- echo "✓ Copied AGENTS.md (agent persona)"
- else
- echo "❌ ERROR: config/AGENTS.md not found"
- exit 1
- fi
- fi
- # Copy skills
- if [ -d "$SCRIPT_DIR/config/skills" ]; then
- mkdir -p "$INSTALL_DIR/skills"
- for skill in "$SCRIPT_DIR/config/skills"/*; do
- skill_name=$(basename "$skill")
- # If --langsmith is provided, only install skills prefixed with "langsmith-"
- if [ "$LANGSMITH_ONLY" = true ] && [[ ! "$skill_name" == langsmith-* ]]; then
- continue
- fi
- if [ -d "$INSTALL_DIR/skills/$skill_name" ]; then
- if [ "$FORCE" = true ]; then
- rm -rf "$INSTALL_DIR/skills/$skill_name"
- else
- echo "⚠️ Skipping $skill_name (already exists, use --force to overwrite)"
- continue
- fi
- fi
- cp -r "$skill" "$INSTALL_DIR/skills/$skill_name"
- echo "✓ Installed $skill_name"
- done
- else
- echo "❌ ERROR: config/skills directory not found"
- exit 1
- fi
- echo ""
- echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
- echo "✅ Installation complete!"
- echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
- echo ""
- echo "Skills installed to $TOOL_NAME!"
- echo ""
- echo "To see installed skills:"
- echo " ls $INSTALL_DIR/skills/"
- echo ""
- if [ "$TARGET" = "deepagents" ]; then
- if [ "$GLOBAL" = true ]; then
- echo "To use this agent, run:"
- echo " deepagents --agent langsmith_agent"
- echo ""
- fi
- echo "For usage, configuration, and API key setup, see the CLI docs:"
- echo " https://docs.langchain.com/deepagents-cli"
- else
- echo "For usage, configuration, and API key setup, see the Claude Code docs:"
- echo " https://code.claude.com/docs/en/overview"
- fi
- echo ""
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.