Skip to content
Back to search

mcp-interviewer

Catch MCP server issues before your agents do.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/microsoft/mcp-interviewer with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/microsoft/mcp-interviewer'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/microsoft--mcp-interviewer/badge.svg)](/listings/microsoft--mcp-interviewer)

HTML

<a href="/listings/microsoft--mcp-interviewer"><img src="/api/listings/microsoft--mcp-interviewer/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

As a CLI tool

pip install mcp-interviewer

# Then,
mcp-interviewer <your mcp server command>

Read more about [CLI usage](./README.md#cli).

As a dependency

Via `uv`

uv add mcp-interviewer

Via `pip`

pip install mcp-interviewer

Read more about [Python usage](./README.md#python).

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 305 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <div align="center">
  2. <h1>MCP Interviewer</h1>
  3. <div>
  4. <i>
  5. A Python CLI tool that helps you catch MCP server issues before your agents do.
  6. </i>
  7. </div>
  8. <a href="https://pypi.org/project/mcp-interviewer/">PyPi</a> | <a href="https://www.microsoft.com/en-us/research/blog/tool-space-interference-in-the-mcp-era-designing-for-agent-compatibility-at-scale/">Blog</a> | <a href="./mcp-interview.md">Example</a>
  9. </div>
  10. ---
  11. ## Table of Contents
  12. - [How it works](#how-it-works)
  13. - [🔎 Constraint checking](#-constraint-checking)
  14. - [🛠️ Functional testing](#️-functional-testing)
  15. - [🤖 LLM evaluation](#-llm-evaluation)
  16. - [📋 Report generation](#-report-generation)
  17. - [Quick Start](#quick-start)
  18. - [Installation](#installation)
  19. - [Example](#example)
  20. - [Usage](#usage)
  21. - [CLI](#cli)
  22. - [Bring Your Own Models](#bring-your-own-models)
  23. - [Python](#python)
  24. - [Limitations](#limitations)
  25. - [Contributing](#contributing)
  26. - [Trademarks](#trademarks)
  27. ## How it works
  28. ### 🔎 Constraint checking
  29. The MCP Interviewer helps avoid violating providers' hard constraints and warns you when you're not following recommended guidance.
  30. For example, OpenAI does not allow more than 128 tools in a single chat completion request, and recommends at most 20 tools.
  31. <details>
  32. <summary>View all supported constraints</summary>
  33. Use `--constraints [CODE ...]` to customize output.
  34. | Constraint Code | Description |
  35. |------------|-------------|
  36. | `OTC` | OpenAI tool count limit (≤128 tools) |
  37. | `ONL` | OpenAI name length (≤64 chars) |
  38. | `ONP` | OpenAI name pattern (a-zA-Z0-9_-) |
  39. | `OTL` | OpenAI token length limit |
  40. | `OA` | All OpenAI constraints |
  41. </details>
  42. ### 🛠️ Functional testing
  43. MCP servers are intended to be used by LLM agents, so the interviewer can optionally test them with an LLM agent. When enabled with the `--test` flag, the interviewer uses your specified LLM to generate a test plan based on the MCP server's capabilities and then executes that plan (e.g. by calling tools), collecting statistics about observed tool behavior.
  44. ### 🤖 LLM evaluation
  45. ***Note: this is an experimental feature. All LLM generated evaluations should be manually inspected for errors.***
  46. The interviewer can also use your specified LLM to provide structured and natural language evaluations of the server's features.
  47. ### 📋 Report generation
  48. The interviewer generates a Markdown report (and accompanying `.json` file with raw data) summarizing the interview results.
  49. <details>
  50. <summary>View all supported reports</summary>
  51. Use `--reports [CODE ...]` to customize output.
  52. | Report Code | Description |
  53. |-------------|-------------|
  54. | `II` | Interviewer Info (model, parameters) |
  55. | `SI` | Server Info (name, version, capabilities) |
  56. | `CAP` | Capabilities (supported features) |
  57. | `TS` | Tool Statistics (counts, patterns) |
  58. | `TCS` | Tool Call Statistics (performance metrics) |
  59. | `FT` | Functional Tests (tool execution results) |
  60. | `CV` | Constraint Violations |
  61. | `T` | Tools |
  62. | `R` | Resources |
  63. | `RT` | Resource Templates |
  64. | `P` | Prompts |
  65. </details>
  66. ## Installation
  67. ### As a CLI tool
  68. ```bash
  69. pip install mcp-interviewer
  70. # Then,
  71. mcp-interviewer <your mcp server command>
  72. ```
  73. Read more about [CLI usage](./README.md#cli).
  74. ### As a dependency
  75. Via `uv`
  76. ```bash
  77. uv add mcp-interviewer
  78. ```
  79. Via `pip`
  80. ```bash
  81. pip install mcp-interviewer
  82. ```
  83. Read more about [Python usage](./README.md#python).
  84. ## Quick Start
  85. ⚠️ ***mcp-interviewer executes the provided MCP server command in a child process. Whenever possible, run your server in a container like in the examples below to isolate the server from your host system.***
  86. First, [install](./README.md#as-a-cli-tool) `mcp-interviewer` as a CLI tool.
  87. ```bash
  88. # Command to run npx safely inside a Docker container
  89. NPX_CONTAINER="docker run -i --rm node:lts npx"
  90. # Interview the MCP reference server
  91. mcp-interviewer \
  92. "$NPX_CONTAINER -y @modelcontextprotocol/server-everything"
  93. ```
  94. Generates a report Markdown `mcp-interview.md` and corresponding JSON data `mcp-interview.json`.
  95. ## Example
  96. To interview the MCP reference server with constraint checking and functional testing you can run the following command:
  97. ```bash
  98. NPX_CONTAINER="docker run -i --rm node:lts npx"
  99. mcp-interviewer --test --model gpt-4.1 "$NPX_CONTAINER -y @modelcontextprotocol/server-everything"
  100. ```
  101. Which will generate a report like [this](./mcp-interview.md).
  102. ## Usage
  103. ### CLI
  104. **Key Flags:**
  105. - `--constraints [CODE ...]`: Customize which constraints to check
  106. - `--reports [CODE ...]`: Customize which report sections to include
  107. - `--test`: Enable functional testing. 🚨 ***This option causes mcp-interviewer to invoke the server's tools. Be careful to limit the server's access to your host system, sensitive data, etc before using these options.***
  108. - `--judge-tools`: Enable experimental LLM evaluation of tools
  109. - `--judge-test`: Enable experimental LLM evaluation of functional tests (requires `--test`)
  110. - `--judge`: Enable all LLM evaluation (equivalent to `--judge-tools --judge-test`)
  111. ```bash
  112. # Docker command to run uvx inside a container
  113. UVX_CONTAINER="docker run -i --rm ghcr.io/astral-sh/uv:python3.12-alpine uvx"
  114. # Basic constraint checking, server inspection, and report generation (no --model needed)
  115. mcp-interviewer "$UVX_CONTAINER mcp-server-fetch"
  116. # Add functional testing with --test (requires --model)
  117. mcp-interviewer --model gpt-4.1 --test "$UVX_CONTAINER mcp-server-fetch"
  118. # Add LLM tool evaluation with --judge-tools (requires --model)
  119. mcp-interviewer --model gpt-4.1 --judge-tools "$UVX_CONTAINER mcp-server-fetch"
  120. # Add LLM test evaluation with --judge-test (requires --model and --test)
  121. mcp-interviewer --model gpt-4.1 --test --judge-test "$UVX_CONTAINER mcp-server-fetch"
  122. # Add all LLM evaluation with --judge (requires --model and --test)
  123. mcp-interviewer --model gpt-4.1 --test --judge "$UVX_CONTAINER mcp-server-fetch"
  124. # Customize report sections with --reports
  125. mcp-interviewer --model gpt-4.1 --test --reports SI TS FT CV "$UVX_CONTAINER mcp-server-fetch"
  126. # Customize constraint checking with --constraints
  127. mcp-interviewer --constraints OTC ONL "$UVX_CONTAINER mcp-server-fetch"
  128. # Fail on constraint warnings for CI/CD pipelines
  129. mcp-interviewer --fail-on-warnings "$UVX_CONTAINER mcp-server-fetch"
  130. # Test remote servers
  131. mcp-interviewer "https://my-mcp-server.com/sse"
  132. ```
  133. ### Bring Your Own Models
  134. MCP Interviewer can use any Python object that mimics the chat completions API of the OpenAI Python SDK's `OpenAI` client.
  135. The CLI provides two ways of customizing your model client:
  136. 1. `openai.OpenAI` keyword arguments
  137. You can provide keyword arguments to the OpenAI client constructor via the "--client-kwargs" CLI option. For example, to connect to gpt-oss:20b running locally via Ollama for LLM features:
  138. ```bash
  139. mcp-interviewer \
  140. --client-kwargs \
  141. "base_url=http://localhost:11434/v1" \
  142. "api_key=ollama" \
  143. --model "gpt-oss:20b" \
  144. --test \
  145. "docker run -i --rm node:lts npx -y @modelcontextprotocol/server-everything"
  146. ```
  147. 1. Import custom `openai.OpenAI`-compatible type
  148. Define a parameterless callable the returns an OpenAI compatible type, then specify it's import path via the "--client" option:
  149. ```python
  150. # my_client.py
  151. from openai import AzureOpenAI
  152. def azure_client():
  153. return AzureOpenAI(azure_endpoint=..., azure_ad_token_provider=...)
  154. ```
  155. ```bash
  156. mcp-interviewer \
  157. --client "my_client.azure_client" \
  158. --model "gpt-4.1_2024-11-20" \
  159. --test \
  160. "docker run -i --rm node:lts npx -y @modelcontextprotocol/server-everything"
  161. ```
  162. ### Python
  163. **Basic usage (constraint checking and server inspection only):**
  164. ```python
  165. from mcp_interviewer import MCPInterviewer, StdioServerParameters
  166. params = StdioServerParameters(
  167. command="docker",
  168. args=["run", "-i", "--rm", "node:lts", "npx", "-y", "@modelcontextprotocol/server-everything"]
  169. )
  170. # No client or model needed for basic functionality
  171. interviewer = MCPInterviewer(None, None)
  172. interview = await interviewer.interview_server(params)
  173. ```
  174. **With LLM features (functional testing and evaluation):**
  175. ```python
  176. from openai import OpenAI
  177. from mcp_interviewer import MCPInterviewer, StdioServerParameters
  178. # Any object following the OpenAI chat completions API will work
  179. client = OpenAI()
  180. params = StdioServerParameters(
  181. command="docker",
  182. args=["run", "-i", "--rm", "node:lts", "npx", "-y", "@modelcontextprotocol/server-everything"]
  183. )
  184. interviewer = MCPInterviewer(client, "gpt-4.1", should_run_functional_test=True)
  185. interview = await interviewer.interview_server(params)
  186. ```
  187. **Using the main function directly (includes constraint checking and report generation):**
  188. ```python
  189. from mcp_interviewer import main, StdioServerParameters
  190. params = StdioServerParameters(
  191. command="docker",
  192. args=["run", "-i", "--rm", "node:lts", "npx", "-y", "@modelcontextprotocol/server-everything"]
  193. )
  194. # Basic usage - no client or model needed
  195. exit_code = main(None, None, params)
  196. # With LLM features
  197. from openai import OpenAI
  198. client = OpenAI()
  199. exit_code = main(client, "gpt-4.1", params, should_run_functional_test=True)
  200. ```
  201. ## Limitations
  202. MCP Interviewer was developed for research and experimental purposes. Further testing and validation are needed before considering its application in commercial or real-world scenarios.
  203. The MCP Python SDK executes arbitrary commands on the host machine, so users should run server commands in isolated containers and use external security tools to validate MCP server safety before running MCP Interviewer.
  204. Additionally, MCP Servers may have malicious or misleading tool metadata that may cause inaccurate MCP Interviewer outputs. Users should manually examine MCP Interviewer outputs for signs of malicious manipulation.
  205. See [TRANSPARENCY.md](./TRANSPARENCY.md) for more information.
  206. ## Contributing
  207. See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on adding new statistics, constraints, and reports.
  208. ## Trademarks
  209. This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft’s Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party’s policies.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.