Skip to content
Back to search

mcp-client-for-ollama

Harness the power of local LLMs with this TUI MCP Client for Ollama. Featuring all core MCP primitives (tools, prompts, resources), agent mode, multi-server, model switching, streaming responses, human-in-the-loop, thinking mode, model params config, system prompts, and saved preferences.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/jonigl--mcp-client-for-ollama/badge.svg)](/listings/jonigl--mcp-client-for-ollama)

HTML

<a href="/listings/jonigl--mcp-client-for-ollama"><img src="/api/listings/jonigl--mcp-client-for-ollama/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Install `ollmcp` via pip, add an MCP server, and run the client:

# Install ollmcp via uv
uv tool install --upgrade ollmcp
# or via pip
pip install --upgrade ollmcp
# Add an MCP server (example: playwright stdio server)
ollmcp mcp add playwright -- npx @playwright/mcp@latest
# Run the client (check optional flags with `ollmcp --help`)
ollmcp # once running, use /help for interactive commands

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 1,310 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <p align="center">
  2. <img src="https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/ollmcp-logo-512.png?raw=true" width="256" />
  3. </p>
  4. <p align="center">
  5. <i>A simple yet powerful Python client for interacting with Model Context Protocol (MCP) servers using Ollama, allowing you to harness local LLMs for advanced tool execution.</i>
  6. </p>
  7. <p align="center">
  8. <b>English</b> | <a href="README.zh-CN.md">简体中文</a> | <a href="README.es.md">Español</a>
  9. </p>
  10. ---
  11. # MCP Client for Ollama (ollmcp)
  12. ![Pepy Total Downloads](https://img.shields.io/pepy/dt/mcp-client-for-ollama?label=Downloads)
  13. [![Python 3.11+](https://img.shields.io/badge/Python-3.11+-blue.svg)](https://www.python.org/downloads/)
  14. [![PyPI - Python Version](https://img.shields.io/pypi/v/ollmcp?label=ollmcp)](https://pypi.org/project/ollmcp/)
  15. [![PyPI - Python Version](https://img.shields.io/pypi/v/mcp-client-for-ollama?label=mcp-client-for-ollama)](https://pypi.org/project/mcp-client-for-ollama/)
  16. [![CI](https://github.com/jonigl/mcp-client-for-ollama/actions/workflows/ci.yml/badge.svg)](https://github.com/jonigl/mcp-client-for-ollama/actions/workflows/ci.yml)
  17. <p align="center">
  18. <sub>Sponsored by</sub>
  19. <br>
  20. <a href="https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=mcp-client-for-ollama">
  21. <picture>
  22. <source media="(prefers-color-scheme: dark)" srcset="https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/atlascloud-logo-dark.png?raw=true">
  23. <img src="https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/atlascloud-logo.png?raw=true" alt="Atlas Cloud" height="20">
  24. </picture>
  25. </a>
  26. <br>
  27. <sub>Learn how to use Atlas Cloud with ollmcp in the <a href="#sponsors">Sponsors</a> section</sub>
  28. </p>
  29. <p align="center">
  30. <img src="https://raw.githubusercontent.com/jonigl/mcp-client-for-ollama/v0.27.0/misc/ollmcp-demo.gif" alt="MCP Client for Ollama Demo">
  31. </p>
  32. <p align="center">
  33. <a href="https://asciinema.org/a/875917" target="_blank">🎥 Watch this demo as an Asciinema recording</a>
  34. </p>
  35. ## Table of Contents
  36. - [Overview](#overview)
  37. - [Features](#features)
  38. - [Requirements](#requirements)
  39. - [Quick Start](#quick-start)
  40. - [Installation Options](#installation-options)
  41. - [Troubleshooting](#troubleshooting)
  42. - ✨**NEW** [Managing MCP Servers via CLI](#managing-mcp-servers-via-cli)
  43. - [mcp add options](#mcp-add-options)
  44. - [Scopes](#scopes)
  45. - [Command-line Arguments](#command-line-arguments)
  46. - [MCP Server Configuration](#mcp-server-configuration)
  47. - ✨**NEW** [Inference Provider Configuration](#inference-provider-configuration)
  48. - [General Options](#general-options)
  49. - ✨**NEW** [Supported Inference Providers](#supported-inference-providers)
  50. - [API key resolution order](#api-key-resolution-order)
  51. - [Usage Examples](#usage-examples)
  52. - [How Tool Calls Work](#how-tool-calls-work)
  53. - [Agent Mode](#agent-mode)
  54. - [Interactive Commands](#interactive-commands)
  55. - [MCP Tools](#mcp-tools)
  56. - [MCP Prompts](#mcp-prompts)
  57. - [MCP Resources](#mcp-resources)
  58. - ✨**NEW** [Answer Display Modes](#answer-display-modes)
  59. - [Input Mode](#input-mode)
  60. - [Model Selection](#model-selection)
  61. - [Advanced Model Configuration](#advanced-model-configuration)
  62. - ✨**NEW** [Thinking Mode and Reasoning Effort](#thinking-mode-and-reasoning-effort)
  63. - [Server Reloading for Development](#server-reloading-for-development)
  64. - [Human-in-the-Loop (HIL) Tool Execution](#human-in-the-loop-hil-tool-execution)
  65. - [Human-in-the-Loop (HIL) Configuration](#human-in-the-loop-hil-configuration)
  66. - [Performance Metrics](#performance-metrics)
  67. - [History Management](#history-management)
  68. - [Autocomplete and Prompt Features](#autocomplete-and-prompt-features)
  69. - [Typer Shell Autocompletion](#typer-shell-autocompletion)
  70. - [FZF-style Autocomplete](#fzf-style-autocomplete)
  71. - [MCP Prompts Autocomplete](#mcp-prompts-autocomplete)
  72. - [Contextual Prompt](#contextual-prompt)
  73. - [Configuration Management](#configuration-management)
  74. - ✨**NEW** [Per-provider profiles](#per-provider-profiles)
  75. - [Server Configuration Format](#server-configuration-format)
  76. - [Tips: Where to Put MCP Server Configs and a Working Example](#tips-where-to-put-mcp-server-configs-and-a-working-example)
  77. - [Compatible Models](#compatible-models)
  78. - [Ollama Cloud Models](#ollama-cloud-models)
  79. - ✨**NEW** [Sponsors](#sponsors)
  80. - [Atlas Cloud](#atlas-cloud)
  81. - [Become a sponsor](#become-a-sponsor)
  82. - [Where Can I Find More MCP Servers?](#where-can-i-find-more-mcp-servers)
  83. - [Related Projects](#related-projects)
  84. - [Security](#security)
  85. - [License](#license)
  86. - [Acknowledgments](#acknowledgments)
  87. ## Overview
  88. MCP Client for Ollama (ollmcp) is a modern, interactive terminal application (TUI) built for harness engineering, connecting local Ollama LLMs to one or more Model Context Protocol (MCP) servers. By fully supporting the core MCP primitives (tools, prompts, and resources), it provides a controlled terminal space where you steer, and the agent executes. With a rich, user-friendly interface, it lets you safely manage your setup in real time with no coding required. Whether you're building, testing, or exploring, this client streamlines your workflow with features like fuzzy autocomplete, advanced model configuration, MCP server hot-reloading for rapid development, and strict Human-in-the-Loop safety controls.
  89. ## Features
  90. - 🤖 **Agent Mode**: Iterative tool execution when models request multiple tool calls, with a configurable loop limit and interactive choices when the limit is reached (continue, wrap up, or abort)
  91. - 🌐 **Multi-Server Support**: Connect to multiple MCP servers simultaneously
  92. - 🚀 **Multiple Transport Types**: Supports STDIO, SSE, and Streamable HTTP server connections
  93. - 📋 **MCP Prompts Support**: Browse, invoke, and manage prompts from MCP servers with argument collection, preview, and safe rollback
  94. - 📦 **MCP Resources Support**: Browse and read contextual data from MCP servers including files, documents, and structured data
  95. - ☁️ **Ollama Cloud Support**: Works seamlessly with Ollama Cloud models for tool calling, enabling access to powerful cloud-hosted models while using local MCP tools
  96. - 🌍 **Multiple LLM Providers**: Use Ollama (default) or OpenAI-compatible providers (OpenAI, OpenRouter, DeepSeek, etc.), with connection settings remembered per provider
  97. - 🎨 **Rich Terminal Interface**: Interactive console UI with modern styling
  98. - 🌊 **Streaming Responses**: View model outputs in real-time as they're generated
  99. - 📝 **Answer Display Modes**: Switch between Plain, Markdown, Both, or Markdown (blocks) response views while streaming
  100. - 🛠️ **Tool Management**: Enable/disable specific tools or entire servers during chat sessions
  101. - 🧑‍💻 **Human-in-the-Loop (HIL)**: Review and approve tool executions before they run for enhanced control and safety
  102. - 🎮 **Advanced Model Configuration**: Fine-tune 15+ model parameters including context window size, temperature, sampling, repetition control, and more
  103. - 💬 **System Prompt Customization**: Define and edit the system prompt to control model behavior and persona
  104. - 🧠 **Context Window Control**: Adjust the context window size (num_ctx) to handle longer conversations and complex tasks
  105. - 🎨 **Enhanced Tool Display**: Beautiful, structured visualization of tool executions with JSON syntax highlighting
  106. - 🧠 **Context Management**: Control conversation memory with configurable retention settings
  107. - 🤔 **Thinking Mode**: Advanced reasoning capabilities with visible thought processes for supported models (e.g., gpt-oss, deepseek-r1, qwen3, etc.)
  108. - 💪 **Reasoning Effort Levels**: Set reasoning effort to auto, minimal, low, medium, high, or xhigh for supported models
  109. - 🖼️ **Vision Tool Support**: Images returned by tools are automatically forwarded to vision-capable models
  110. - 🗣️ **Cross-Language Support**: Seamlessly work with both Python and JavaScript MCP servers
  111. - 📜 **History Management**: View full conversation history, export to JSON for backup/analysis, and import previous sessions for continuity
  112. - 🔍 **Auto-Discovery**: Automatically find and use Claude's existing MCP server configurations
  113. - 🔁 **Dynamic Model Switching**: Switch between any installed Ollama model without restarting
  114. - 💾 **Configuration Persistence**: Save and load tool preferences and model settings between sessions
  115. - 🔄 **Server Reloading**: Hot-reload MCP servers during development without restarting the client
  116. - ✨ **Fuzzy Autocomplete**: Interactive, arrow-key command autocomplete with descriptions
  117. - 🏷️ **Dynamic Prompt**: Shows current model, thinking mode, and enabled tools
  118. - 📊 **Performance Metrics**: Detailed model performance data after each query, including duration timings and token counts
  119. - 🔌 **Plug-and-Play**: Works immediately with standard MCP-compliant tool servers
  120. - 🔔 **Update Notifications**: Automatically detects when a new version is available
  121. - 🖥️ **Modern CLI with Typer**: Grouped options, shell autocompletion, and improved help output
  122. - ⏹️ **Abort Generation**: You can abort model generation at any time by pressing 'a' during response streaming
  123. ## Requirements
  124. - **Python 3.11+** ([Installation guide](https://www.python.org/downloads/))
  125. - **Ollama** running locally ([Installation guide](https://ollama.com/download))
  126. - After installation, run `ollama list` to see available models. If no models are installed, you can pull one using `ollama pull <model_name>`. For example, `ollama pull gemma4:latest`.
  127. - **UV package manager** ([Installation guide](https://github.com/astral-sh/uv))
  128. ## Quick Start
  129. Install `ollmcp` via pip, add an MCP server, and run the client:
  130. ```bash
  131. # Install ollmcp via uv
  132. uv tool install --upgrade ollmcp
  133. # or via pip
  134. pip install --upgrade ollmcp
  135. # Add an MCP server (example: playwright stdio server)
  136. ollmcp mcp add playwright -- npx @playwright/mcp@latest
  137. # Run the client (check optional flags with `ollmcp --help`)
  138. ollmcp # once running, use /help for interactive commands
  139. ```
  140. ## Installation Options
  141. **Option 1:** Install with uv and run (recommended)
  142. ```bash
  143. uv tool install --upgrade ollmcp
  144. ollmcp
  145. ```
  146. **Option 2:** Install with pip and run
  147. ```bash
  148. pip install --upgrade ollmcp
  149. ollmcp
  150. ```
  151. **Option 3:** Only run without installing (requires `uv` package manager)
  152. ```bash
  153. uvx ollmcp
  154. ```
  155. **Option 4:** Install from source and run using virtual environment
  156. ```bash
  157. git clone https://github.com/jonigl/mcp-client-for-ollama.git
  158. cd mcp-client-for-ollama
  159. uv run -m mcp_client_for_ollama
  160. ```
  161. ## Troubleshooting
  162. ### `Could not find a version that satisfies the requirement ollmcp (from versions: none)`
  163. This almost always means the Python you are using is **older than the required 3.11+**. This is common on macOS, where the system Python (`/usr/bin/python3`) or the Xcode-bundled Python can be 3.9 or older. When no release matches `requires-python >= 3.11`, pip filters out every version and reports the misleading "from versions: none".
  164. First check your version:
  165. ```bash
  166. python3 --version # must be 3.11 or newer
  167. ```
  168. Then install with a modern Python. The simplest option is `uv`, which fetches a suitable Python for you automatically:
  169. ```bash
  170. uv tool install --upgrade ollmcp # recommended, installs the CLI in an isolated environment
  171. # or, if you prefer pip, make sure to use a Python 3.11+ interpreter:
  172. python3.11 -m pip install --upgrade ollmcp
  173. # Then run the client:
  174. ollmcp
  175. ```
  176. Take a look at the [Installation Options](#installation-options).
  177. ### `error: externally-managed-environment` (PEP 668)
  178. On recent Debian/Ubuntu (Python 3.12+), the system `pip` is intentionally locked to protect OS-managed packages, so `pip install ollmcp` is blocked. This is a system policy ([PEP 668](https://peps.python.org/pep-0668/)), not an issue with ollmcp. Install it into an isolated environment instead:
  179. ```bash
  180. uv tool install --upgrade ollmcp # recommended, installs the CLI in an isolated environment
  181. # or, if you prefer pip, use a virtual environment:
  182. python3.11 -m venv ollmcp-env
  183. source ollmcp-env/bin/activate
  184. python3.11 -m pip install --upgrade ollmcp
  185. # Then run the client:
  186. ollmcp
  187. ```
  188. Take a look at the [Installation Options](#installation-options).
  189. > [!WARNING]
  190. > Avoid `pip install --break-system-packages ollmcp`. It works, but it installs into the system Python and can break packages your OS depends on.
  191. ## Managing MCP Servers via CLI
  192. ollmcp can manage its own MCP server configurations directly from the command line, similar to `claude mcp`:
  193. ```bash
  194. # Remote servers (Streamable HTTP or SSE)
  195. ollmcp mcp add --transport http <name> <url>
  196. ollmcp mcp add --transport sse <name> <url>
  197. # Local stdio servers - everything after `--` is the command to run
  198. ollmcp mcp add [options] <name> -- <command> [args...]
  199. # List configured servers
  200. ollmcp mcp list
  201. # Remove a server
  202. ollmcp mcp remove <name>
  203. # For more details on options and usage, run:
  204. ollmcp mcp --help
  205. ollmcp mcp add --help
  206. ```
  207. **Examples:**
  208. > [!TIP]
  209. > Once you have added some servers, simply running `ollmcp` will connect to them automatically.
  210. ```bash
  211. ollmcp mcp add --transport http github https://api.githubcopilot.com/mcp/ --header "Authorization: Bearer $YOUR_GITHUB_PAT"
  212. ollmcp mcp add --transport stdio playwright npx @playwright/mcp@latest
  213. ollmcp mcp add filesystem -- npx -y @modelcontextprotocol/server-filesystem /allowed-dir1 ~/allowed-dir2 # stdio transport by default
  214. ollmcp mcp add --env API_KEY=YOUR_KEY --transport sse my-sse-server http://localhost:8000/sse
  215. ```
  216. ### mcp add options
  217. - `--transport`, `-t`: `stdio` (default), `sse`, or `http`.
  218. - `--header`, `-H`: HTTP header as `"Name: Value"` for `sse`/`http` servers. Repeatable.
  219. - `--env`, `-e`: Environment variable as `KEY=value` for `stdio` servers. Repeatable.
  220. - `--scope`, `-s`: Where to store the server (see scopes below). Default: `local`.
  221. ### Scopes
  222. | Scope | Loads in | Shared with team | Stored in |
  223. |-----------|-----------------------|-------------------|-----------|
  224. | `local` | Current project only | No | `~/.config/ollmcp/mcp.local.json` (keyed by project path) |
  225. | `project` | Current project only | Yes (via VCS) | `.mcp.json` in the project root |
  226. | `user` | All your projects | No | `~/.config/ollmcp/mcp.json` |
  227. The `project` scope writes a standard `.mcp.json` file at your project root, compatible with Claude Code and other MCP-aware tools. If the same server name exists in multiple scopes, precedence is `local` > `project` > `user`.
  228. > [!NOTE]
  229. > Servers added via `ollmcp mcp add` are always loaded as the base layer. Any flags (`--mcp-server`, `--mcp-server-url`, `--servers-json`, `--claude-desktop`) add on top. To include servers from Claude Desktop, pass `--claude-desktop` explicitly.
  230. >
  231. > If a server with the same name is also provided via one of those flags, both connections are currently opened, but only one is kept active under that name. Avoid reusing a registry server's name in `--mcp-server`/`--mcp-server-url`/`--servers-json`/`--claude-desktop`.
  232. ### Command-line Arguments
  233. > [!TIP]
  234. > The CLI now uses `Typer` for a modern experience: grouped options, rich help, and built-in shell autocompletion. Advanced users can use short flags for faster commands. To enable autocompletion, run:
  235. >
  236. > ```bash
  237. > ollmcp --install-completion
  238. > ```
  239. >
  240. > Then restart your shell or follow the printed instructions.
  241. #### MCP Server Configuration:
  242. - `--mcp-server`, `-s`: Path to one or more MCP server scripts (.py or .js). Can be specified multiple times.
  243. - `--mcp-server-url`, `-u`: URL to one or more SSE or Streamable HTTP MCP servers. Can be specified multiple times. See [Common MCP endpoint paths](#common-mcp-endpoint-paths) for typical endpoints.
  244. - `--servers-json`, `-j`: Path to a JSON file with server configurations. See [Server Configuration Format](#server-configuration-format) for details.
  245. - `--claude-desktop`: Load servers from Claude Desktop's config file (`~/Library/Application Support/Claude/claude_desktop_config.json`). Merged with servers added via `ollmcp mcp add` and any other flags.
  246. > [!IMPORTANT]
  247. > **Breaking change:** `--auto-discovery` / `-a` has been replaced by `--claude-desktop`. Additionally, servers added via `ollmcp mcp add` are now always loaded automatically, they are no longer a fallback that disappears when other flags are used. Claude Desktop servers are never loaded automatically; use `--claude-desktop` to include them.
  248. #### Inference Provider Configuration:
  249. - `--model`, `-m` MODEL: Model to use. Default: your saved configuration's model if set, otherwise the first model available in Ollama
  250. - `--provider`, `-p` PROVIDER: LLM provider to use (e.g. `ollama`, `openai`, `atlascloud`, `openrouter`, `deepseek`). Default: `ollama`
  251. - `--host`, `-H` HOST: LLM host / API base URL. Defaults to Ollama's `http://localhost:11434` for the `ollama` provider, or the provider's own default endpoint otherwise.
  252. - `--api-key`, `-k` KEY: API key for the LLM provider. Also read from the `$OLLMCP_API_KEY` environment variable, which is **provider-agnostic** (it applies to whichever provider you select with `--provider`). Keys passed via `$OLLMCP_API_KEY` are never written to the config file; only keys passed with `--api-key` are saved. Not needed for `ollama`.
  253. > [!NOTE]
  254. > Currently supported providers: `ollama`, `openai`, `atlascloud`, and any OpenAI-compatible provider (`openrouter`, `deepseek`, `perplexity`, etc.). More providers coming soon.
  255. #### General Options:
  256. - `--version`, `-v`: Show version and exit
  257. - `--help`, `-h`: Show help message and exit
  258. - `--install-completion`: Install shell autocompletion scripts for the client
  259. - `--show-completion`: Show available shell completion options
  260. #### MCP Server Logging:
  261. Whatever an MCP server reports is written to `~/.config/ollmcp/logs/<session>/<server>.log`, one directory per run, keeping the last 5. **Nothing a server prints reaches the screen unless you ask for it** — otherwise it would draw over the answer being streamed.
  262. Servers report through two channels: their **stderr** (only stdio servers, since a remote one runs elsewhere) and **MCP log notifications** (any server). **Both flags only change what you see on screen — the log file gets everything either way:**
  263. | | written to the log file | shown on screen |
  264. |---|---|---|
  265. | *(no flag)* | everything the server sends | nothing from the server |
  266. | `--debug` | everything the server sends | all of it, as it arrives |
  267. | `--log-level LEVEL` | everything the server sends | only notifications of LEVEL or higher |
  268. | `--debug --log-level LEVEL` | everything the server sends | stderr in full + notifications of LEVEL or higher |
  269. LEVEL is one of `debug`, `info`, `notice`, `warning`, `error`, `critical`, `alert`, `emergency`.
  270. With no flag, no level is requested at all: the server decides what it emits and all of it is recorded. `--log-level` filters what you see, and is also sent to the server (`logging/setLevel`) when it advertises the logging capability — such a server may then stop emitting the lower levels, which are missing from the file too. Servers without that capability keep sending everything and the filtering happens here.
  271. When a server fails to connect, the error points at its log file: whatever the server printed on its way out is in there, and that is usually the real reason.
  272. > [!NOTE]
  273. > What ollmcp itself reports goes to `~/.config/ollmcp/logs/<session>/ollmcp.log`, next to the server files. It is written on every run, no `--debug` needed, and it is the file the on-screen warning points at when a response stream ends early. Work in progress: for now it records mostly provider and streaming errors, and more will be logged there over time.
  274. ### Supported Inference Providers
  275. > [!WARNING]
  276. > **Non-Ollama providers are experimental.** Support for providers other than Ollama was added recently and is still being stabilized, not everything may work correctly yet.
  277. ollmcp works with **Ollama** plus any **OpenAI-compatible** provider that [any-llm](https://github.com/mozilla-ai/any-llm) exposes. Select one with `--provider`. Provide the key with `--api-key` or `$OLLMCP_API_KEY` (both work for **any** selected provider) or via the provider's own environment variable shown below. `$OLLMCP_API_KEY` and the provider-native env vars are never written to disk; only a key passed with `--api-key` is saved to the config.
  278. | Provider (`--provider`) | API key env var |
  279. |---|---|
  280. | [`ollama`](https://github.com/ollama/ollama) (default) | - (local) |
  281. | [`atlascloud`](https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=mcp-client-for-ollama) | `ATLASCLOUD_API_KEY` |
  282. | [`azureopenai`](https://learn.microsoft.com/en-us/azure/ai-foundry/) | `AZURE_OPENAI_API_KEY` |
  283. | [`dashscope`](https://bailian.console.aliyun.com/cn-beijing/?tab=api#/api) | `DASHSCOPE_API_KEY` |
  284. | [`databricks`](https://docs.databricks.com/) | `DATABRICKS_TOKEN` |
  285. | [`deepinfra`](https://deepinfra.com/docs/openai_api) | `DEEPINFRA_API_KEY` |
  286. | [`deepseek`](https://platform.deepseek.com/) | `DEEPSEEK_API_KEY` |
  287. | [`fireworks`](https://fireworks.ai/api) | `FIREWORKS_API_KEY` |
  288. | [`gateway`](https://github.com/mozilla-ai/any-llm) | `GATEWAY_API_KEY` |
  289. | [`inception`](https://inceptionlabs.ai/) | `INCEPTION_API_KEY` |
  290. | [`llama`](https://www.llama.com/products/llama-api/) | `LLAMA_API_KEY` |
  291. | [`llamacpp`](https://github.com/ggml-org/llama.cpp) | - (local) |
  292. | [`llamafile`](https://github.com/Mozilla-Ocho/llamafile) | - (local) |
  293. | [`lmstudio`](https://lmstudio.ai/) | `LM_STUDIO_API_KEY` |
  294. | [`minimax`](https://www.minimax.io/platform_overview) | `MINIMAX_API_KEY` |
  295. | [`moonshot`](https://platform.moonshot.ai/) | `MOONSHOT_API_KEY` |
  296. | [`mzai`](https://any-llm.ai) | `ANY_LLM_KEY` |
  297. | [`nebius`](https://studio.nebius.ai/) | `NEBIUS_API_KEY` |
  298. | [`openai`](https://platform.openai.com/docs/api-reference) | `OPENAI_API_KEY` |
  299. | [`openrouter`](https://openrouter.ai/docs) | `OPENROUTER_API_KEY` |
  300. | [`perplexity`](https://docs.perplexity.ai/) | `PERPLEXITY_API_KEY` |
  301. | [`portkey`](https://portkey.ai/docs) | `PORTKEY_API_KEY` |
  302. | [`qiniu`](https://developer.qiniu.com/aitokenapi) | `QINIU_API_KEY` |
  303. | [`sambanova`](https://sambanova.ai/) | `SAMBANOVA_API_KEY` |
  304. | [`vllm`](https://docs.vllm.ai/) | `VLLM_API_KEY` |
  305. | [`zai`](https://docs.z.ai/guides/develop/python/introduction) | `ZAI_API_KEY` |
  306. > [!NOTE]
  307. > Local OpenAI-compatible servers (`ollama`, `llamacpp`, `llamafile`, `lmstudio`, `vllm`) typically run without an API key, point ollmcp at them with `--host`. Providers any-llm offers that are **not** OpenAI-compatible (e.g. `anthropic`, `gemini`, `mistral`, `groq`, `cohere`) are not supported yet.
  308. > [!WARNING]
  309. > **Capability detection limitation:** ollmcp only reads real per-model capabilities (`tools`, `vision`, `thinking`) from Ollama. For every non-Ollama provider, all three capabilities are currently assumed available and shown as such in the model list and badges, so a model may be reported as supporting tools, vision, or thinking even when it doesn't. If a model lacks a capability, the provider's API will return an error when you try to use it.
  310. #### API key resolution order
  311. For the selected provider, ollmcp resolves the API key in this order, from **highest** to **lowest** precedence:
  312. 1. The `--api-key` / `-k` flag.
  313. 2. The `$OLLMCP_API_KEY` environment variable (provider-agnostic, applies to whichever provider you selected with `--provider`).
  314. 3. The per-provider key saved in `~/.config/ollmcp/config.json` (present only if it was once passed via `--api-key`).
  315. 4. The provider's own native environment variable, detected by [any-llm](https://github.com/mozilla-ai/any-llm) (e.g. `OPENAI_API_KEY`, `OPENROUTER_API_KEY`).
  316. > [!WARNING]
  317. > A saved per-provider key (3) takes precedence over the provider's native environment variable (4). So if you previously saved a **wrong or expired** key, setting `OPENAI_API_KEY` (or the equivalent) alone will **not** override it. To fix it, either pass the correct key with `--api-key`, or remove the stale `apiKey` from that provider's profile in `~/.config/ollmcp/config.json`.
  318. ### Usage Examples
  319. Simplest way to run the client:
  320. ```bash
  321. ollmcp
  322. ```
  323. > [!TIP]
  324. > This connects to all servers registered via `ollmcp mcp add` and uses the model from your saved configuration file, or the first available model in Ollama if none is saved. Pass `--claude-desktop` to also include servers from Claude Desktop's config.
  325. Connect to a single server:
  326. ```bash
  327. ollmcp --mcp-server /path/to/weather.py --model llama3.2:3b
  328. # Or using short flags:
  329. ollmcp -s /path/to/weather.py -m llama3.2:3b
  330. ```
  331. Connect to multiple servers:
  332. ```bash
  333. ollmcp --mcp-server /path/to/weather.py --mcp-server /path/to/filesystem.js
  334. # Or using short flags:
  335. ollmcp -s /path/to/weather.py -s /path/to/filesystem.js
  336. ```
  337. > [!TIP]
  338. > If `--model` is not specified, the model from your saved configuration file is used; otherwise the first available model in Ollama is selected automatically (you'll be told how to pull one if none are installed).
  339. Use a JSON configuration file:
  340. ```bash
  341. ollmcp --servers-json /path/to/servers.json --model llama3.2:1b
  342. # Or using short flags:
  343. ollmcp -j /path/to/servers.json -m llama3.2:1b
  344. ```
  345. > [!TIP]
  346. > See the [Server Configuration Format](#server-configuration-format) section for details on how to structure the JSON file.
  347. Use a custom Ollama host:
  348. ```bash
  349. ollmcp --host http://localhost:22545 --servers-json /path/to/servers.json
  350. # Or using short flags:
  351. ollmcp -H http://localhost:22545 -j /path/to/servers.json
  352. ```
  353. Use a different LLM provider (OpenAI or any OpenAI-compatible API):
  354. ```bash
  355. ollmcp --provider openai --api-key $OPENAI_API_KEY --model gpt-5.5
  356. # OpenAI-compatible providers (e.g. OpenRouter, DeepSeek); override the endpoint with --host if needed:
  357. ollmcp --provider openrouter --api-key $OPENROUTER_API_KEY -m openrouter/free
  358. ```
  359. > [!TIP]
  360. > Provider settings (model, host, API key) are remembered **per provider**. Once saved with `/save-config`, plain `ollmcp` resumes your last-used provider. See [Configuration Management](#configuration-management) for details.
  361. Connect to SSE or Streamable HTTP servers by URL:
  362. ```bash
  363. ollmcp --mcp-server-url http://localhost:8000/sse --model qwen2.5:latest
  364. # Or using short flags:
  365. ollmcp -u http://localhost:8000/sse -m qwen2.5:latest
  366. ```
  367. Connect to multiple URL servers:
  368. ```bash
  369. ollmcp --mcp-server-url http://localhost:8000/sse --mcp-server-url http://localhost:9000/mcp
  370. # Or using short flags:
  371. ollmcp -u http://localhost:8000/sse -u http://localhost:9000/mcp
  372. ```
  373. Mix local scripts and URL servers:
  374. ```bash
  375. ollmcp --mcp-server /path/to/weather.py --mcp-server-url http://localhost:8000/mcp --model qwen3:1.7b
  376. # Or using short flags:
  377. ollmcp -s /path/to/weather.py -u http://localhost:8000/mcp -m qwen3:1.7b
  378. ```
  379. Include Claude Desktop servers alongside other sources:
  380. ```bash
  381. ollmcp --mcp-server /path/to/weather.py --mcp-server-url http://localhost:8000/mcp --claude-desktop
  382. # Or using short flags:
  383. ollmcp -s /path/to/weather.py -u http://localhost:8000/mcp --claude-desktop
  384. ```
  385. ### How Tool Calls Work
  386. 1. The client sends your query to Ollama with a list of available tools
  387. 2. If Ollama decides to use a tool, the client:
  388. - Displays the tool execution with formatted arguments and syntax highlighting
  389. - Shows a Human-in-the-Loop confirmation prompt (if enabled) allowing you to review and approve the tool call
  390. - Extracts the tool name and arguments from the model response
  391. - Calls the appropriate MCP server with these arguments (only if approved or HIL is disabled)
  392. - Shows the tool response in a structured, easy-to-read format (including image and unsupported-media summaries)
  393. - If the tool returned images and the current model supports vision, attaches the images to the next LLM message; otherwise displays a warning
  394. - Sends the tool result back to Ollama
  395. - If in Agent Mode, repeats the process if the model requests more tool calls
  396. 3. Finally, the client:
  397. - Displays the model's final response incorporating the tool results
  398. ### Agent Mode
  399. Some models may request multiple tool calls in a single conversation. The client supports an **Agent Mode** that allows for iterative tool execution:
  400. - When the model requests a tool call, the client executes it and sends the result back to the model
  401. - This process repeats until the model provides a final answer or reaches the configured loop limit
  402. - You can set the maximum number of iterations using the `/loop-limit` (`/ll`) command
  403. - The default loop limit is `7` to prevent infinite loops
  404. #### When the loop limit is reached
  405. Instead of silently stopping, the client pauses and asks you how to proceed:
  406. | Choice | Key | Description |
  407. |--------|-----|-------------|
  408. | **Continue** | `c` *(default)* | Grant another batch of iterations (same size as the current limit) |
  409. | **Number** | `n` | Choose exactly how many more iterations to allow |
  410. | **Unlimited** | `u` | Remove the cap and run until the model stops requesting tools |
  411. | **Wrap up** | `w` | Ask the model to summarise what it gathered so far and produce a final answer — preserves all tool results collected before the limit |
  412. | **Abort** | `a` | Discard the turn entirely (nothing saved to history) |
  413. > [!NOTE]
  414. > If you want to prevent using Agent Mode, simply set the loop limit to `1`.
  415. #### Agent Mode Quick Demo:
  416. [![asciicast](https://asciinema.org/a/476qpEamCX9TFQt4jNEXIgHxS.svg)](https://asciinema.org/a/476qpEamCX9TFQt4jNEXIgHxS)
  417. ## Interactive Commands
  418. During chat, use these commands:
  419. > [!IMPORTANT]
  420. > **NEW:** Built-in interactive commands now require a leading `/`.
  421. > - Use `/help`, `/model`, `/tools`, `/prompts`, etc.
  422. > - Bare command names like `help` or `model` are no longer executed as commands.
  423. > - Prompt invocations also use `/`, with `/server:prompt_name` recommended to avoid collisions.
  424. ![ollmcp main interface](https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/ollmcp-welcome.png?raw=true)
  425. | Command | Shortcut | Description |
  426. |------------------|------------------|-----------------------------------------------------|
  427. | `abort` | `a` | While model is generating, abort the current response generation |
  428. | `/clear`, `/new` | `/cc` | Clear conversation history and context |
  429. | `/cls` | `/clear-screen` | Clear the terminal screen |
  430. | `/context` | `/c` | Toggle context retention |
  431. | `/context-info` | `/ci` | Display context statistics |
  432. | `/export-history`| `/eh` | Export chat history to a JSON file |
  433. | `/full-history` | `/fh` | Display all conversation history |
  434. | `/help` | `/h` | Display help and available commands |
  435. | `/import-history`| `/ih` | Import chat history from a JSON file |
  436. | `/human-in-the-loop` | `/hil` | Toggle Human-in-the-Loop confirmations for tool execution |
  437. | `/load-config` | `/lc` | Load tool and model configuration from a file |
  438. | `/loop-limit` | `/ll` | Set maximum iterative tool-loop iterations (Agent Mode). Default: 7 |
  439. | `/model` | `/m` | List and select a different Ollama model |
  440. | `/model-config` | `/mc` | Configure advanced model parameters and system prompt |
  441. | `/display-mode` | `/dm` | Choose Plain, Markdown, Both, or Markdown (blocks) answer display modes |
  442. | `/input-mode` | `/im` | Choose Single-line or Multiline chat input mode |
  443. | `/prompts` | `/pr` | Browse and view all available MCP prompts |
  444. | `/server:prompt_name` | `/prompt_name` | Invoke a prompt (qualified is recommended) |
  445. | `/resources` | `/res` | Browse and view all available MCP resources |
  446. | `@uri` | - | Read a specific resource by URI (e.g., `@server://info`) |
  447. | `/quit`, `/exit`, `/bye` | `/q`, `Ctrl+C`, or `Ctrl+D` | Exit the client |
  448. | `/reload-servers`| `/rs` | Reload all MCP servers with current configuration |
  449. | `/reset-config` | `/rc` | Reset configuration to defaults (all tools enabled) |
  450. | `/save-config` | `/sc` | Save current tool and model configuration to a file |
  451. | `/show-metrics` | `/sm` | Toggle performance metrics display |
  452. | `/show-thinking` | `/st` | Toggle thinking text visibility (visible by default) |
  453. | `/thinking-mode` | `/tm` | Toggle thinking mode on supported models |
  454. | `/reasoning-effort` | `/re` | Set reasoning effort level (auto/minimal/low/medium/high/xhigh) when thinking mode is on. Default: medium |
  455. | `/show-tool-execution` | `/ste` | Toggle tool execution display visibility |
  456. | `/tools` | `/t` | Open the tool selection interface |
  457. ### MCP Tools
  458. The tool and server selection interface allows you to enable or disable specific tools:
  459. ![ollmcp tool and server selection interface](https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/ollmpc-tool-and-server-selection.png?raw=true)
  460. - Enter **numbers** separated by commas (e.g. `1,3,5`) to toggle specific tools
  461. - Enter **ranges** of numbers (e.g. `5-8`) to toggle multiple consecutive tools
  462. - Enter **S + number** (e.g. `S1`) to toggle all tools in a specific server
  463. - `a` or `all` - Enable all tools
  464. - `n` or `none` - Disable all tools
  465. - `d` or `desc` - Show/hide tool descriptions
  466. - `j` or `json` - Show detailed tool JSON schemas on enabled tools for debugging purposes
  467. - `s` or `save` - Save changes and return to chat
  468. - `q` or `quit` - Cancel changes and return to chat
  469. ### MCP Prompts
  470. MCP Prompts provide reusable, server-defined conversation starters and context templates. Servers can expose prompts with descriptions, required arguments, and pre-formatted messages that help you quickly start specific types of conversations or inject structured context into your chat.
  471. #### Features
  472. - 📋 **Browse Prompts**: View all available prompts from connected servers with descriptions and argument requirements
  473. - ⚡️ **Quick Invocation**: Use slash syntax to invoke prompts (`/server:prompt_name` recommended)
  474. - 🔤 **Autocomplete**: Type `/` to see prompt suggestions with fuzzy matching
  475. - 📝 **Argument Collection**: Interactive prompts guide you through required parameters
  476. - 👁️ **Preview**: Review prompt content before injection to ensure it fits your needs
  477. - 🎯 **Flexible Injection**: Choose to execute immediately or inject-only (add to history without triggering model)
  478. - 🧠 **Context-Aware**: Automatically adapts behavior based on whether prompt ends with user or assistant message
  479. - 🔄 **Safe Rollback**: Automatic history cleanup if you abort or encounter errors
  480. - 💬 **Text Content**: Supports text-based prompt messages (image/audio/resource support coming soon)
  481. #### How to Use MCP Prompts
  482. **Browse Available Prompts:**
  483. ```
  484. /prompts # or '/pr'
  485. ```
  486. This displays all prompts grouped by server, showing their names, required arguments, and descriptions.
  487. **Invoke a Prompt:**
  488. ```
  489. /server:prompt_name
  490. ```
  491. For example, if a server named `docs` provides a "summarize" prompt:
  492. ```
  493. /docs:summarize
  494. ```
  495. If a prompt name is unique across connected servers, you can use the short form:
  496. ```
  497. /summarize
  498. ```
  499. If multiple servers expose the same prompt name, the client will ask you to use the qualified form and suggest valid `/server:prompt_name` options.
  500. **Autocomplete:**
  501. - Type `/` to see all available prompts with descriptions
  502. - Continue typing to filter prompts with fuzzy matching
  503. - Use arrow keys to navigate and press Enter to select
  504. > [!TIP]
  505. > Prompts are discovered automatically when you connect to MCP servers. If a server supports prompts, they'll be available immediately in the `prompts` list and autocomplete.
  506. **Workflow:**
  507. 1. Type `/server:prompt_name` (recommended) or select from autocomplete
  508. 2. If the prompt requires arguments, you'll be prompted to provide them
  509. 3. Review the prompt preview showing what will be injected
  510. 4. Choose how to use the prompt:
  511. - **y/yes** (default): Send the prompt to the model and get a response
  512. - For prompts ending with a **user message**: Uses that message as the query
  513. - For prompts ending with an **assistant message**: Adds "Please respond based on the above context." as the query
  514. - **i/inject**: Just add the prompt to conversation history without triggering the model (lets you type your own query afterward)
  515. - **n/no**: Cancel and return to chat
  516. 5. The prompt is injected based on your choice
  517. 6. If you abort during model generation (press 'a'), changes are automatically rolled back
  518. **Example:**
  519. ![ollmcp prompt feature screenshot](https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/ollmcp-prompt-feature.png?raw=true)
  520. > [!WARNING]
  521. > **Content Type Limitations**: MCP Prompts currently support **text content only**. The following content types are not yet supported and will be automatically skipped:
  522. > - 🖼️ **Images** - Image content in prompts
  523. > - 🎵 **Audio** - Audio content in prompts
  524. > - 📦 **Resources** - Embedded resource content
  525. ### MCP Resources
  526. MCP Resources provide access to contextual data exposed by MCP servers-files, documents, structured data, and more. Servers can expose resources with metadata (name, description, MIME type) that you can browse and read into your conversation context.
  527. #### Features
  528. - 📋 **Browse Resources**: View all available resources from connected servers with URIs, names, MIME types, and descriptions
  529. - 📖 **Read Resources**: Use `@uri` syntax to read resource content, standalone or inline within a query
  530. - 📝 **Text Content**: Full support for text-based resources (markdown, code, logs, etc.)
  531. - 🖼️ **Vision Image Support**: Image resources (`image/*`) are automatically forwarded as base64 images to vision-capable models
  532. - 🎯 **Context Injection**: Resource content is buffered and injected as context alongside your next query
  533. - 🔍 **Autocomplete**: Type `@` to see available resource and template suggestions with fuzzy matching
  534. - 🛡️ **Binary Safety**: Non-image binary content (audio, video, PDFs, archives) is detected and gracefully skipped with informative messages
  535. #### How to Use MCP Resources
  536. **Browse Available Resources:**
  537. ```
  538. /resources # or '/res'
  539. ```
  540. This displays all resources and templates grouped by server, showing URIs, names, MIME types, and descriptions. Binary resources are marked with a `[binary]` tag and templates with a `[template]` tag.
  541. **Read a Resource:**
  542. ```
  543. @<uri>
  544. ```
  545. For example, to read a file resource:
  546. ```
  547. @file:///path/to/document.md
  548. ```
  549. There are two ways to use `@uri`:
  550. **1. Standalone (buffer then query):** Type `@uri` on its own. The resource is fetched and buffered. Then type your query on the next prompt. The resource content is injected as context automatically.
  551. **2. Inline (single turn):** Include `@uri` anywhere inside your query text. The resource is fetched and the query is processed immediately in one step.
  552. **Standalone example:**
  553. ```
  554. qwen3/show-thinking/6-tools❯ @server://info
  555. ✅ Read resource 'get_server_info' (197 chars)
  556. Preview:
  557. This is a simple MCP server with streamable HTTP transport. It supports tools for greeting, adding numbers, generating
  558. random numbers, and calculating BMI. It also provides a BMI calculator prompt.
  559. 1 resource(s) buffered. Type your query, or include @another_uri inline.
  560. qwen3/show-thinking/6-tools❯ Next question here
  561. ```
  562. **Inline example:**
  563. ```
  564. qwen3/show-thinking/6-tools❯ summarize the key features from @server://info
  565. ✅ Read resource 'get_server_info' (197 chars)
  566. Preview:
  567. This is a simple MCP server with streamable HTTP transport. It supports tools for greeting, adding numbers, generating
  568. random numbers, and calculating BMI. It also provides a BMI calculator prompt.
  569. [model response]
  570. ```
  571. > [!TIP]
  572. > Resources are discovered automatically when you connect to MCP servers. If a server supports resources, they'll be available immediately in the `resources` list and `@` autocomplete.
  573. > [!NOTE]
  574. > 🖼️ **Images** (`image/*`) **are** supported, they are passed directly to vision-capable models as base64 data.
  575. > **Binary Content**: The following resource types are **not** supported as context and will be skipped with an informative message:
  576. > - 🎵 **Audio** - `audio/*` MIME types
  577. > - 📹 **Video** - `video/*` MIME types
  578. > - 📄 **PDFs** - `application/pdf`
  579. > - 🗜️ **Archives** - `application/zip`, `application/octet-stream`
  580. >
  581. ### Answer Display Modes
  582. The `/display-mode` (`/dm`) command lets you choose how model answers are shown while they stream:
  583. - **Plain**: Streams the response once as plain text with no final markdown re-render
  584. - ✨**NEW** **Markdown** (default): Streams formatted markdown line by line — lines above a small live tail are printed once and never redrawn, so it stays reliable even with emojis or terminal resizes
  585. - **Both**: Streams plain text first, then renders the completed response again as markdown
  586. - **Markdown (blocks)**: Renders the response as markdown one block at a time, append-only each paragraph/list/table/code block prints once when it completes and is never redrawn, so it cannot duplicate lines
  587. Use `/display-mode` or `/dm` during chat to open the interactive picker.
  588. **Why you might switch modes:**
  589. - **Plain** is the least noisy option if you want minimal redraw or flicker
  590. - **Markdown** combines line-by-line streaming with coherent markdown formatting; at most the last few lines are ever redrawn, so glitches stay bounded even with emojis or resizes
  591. - **Both** gives you fast streaming feedback plus a clean final markdown rendering
  592. - **Markdown (blocks)** is the most conservative way to see formatted markdown while streaming, at the cost of block-by-block (rather than line-by-line) updates
  593. > [!TIP]
  594. > Your selected display mode is saved with `/save-config` and restored with `/load-config`, so you can keep different viewing preferences for different workflows.
  595. ### Input Mode
  596. The `/input-mode` (`/im`) command controls how you write chat messages:
  597. - **Single-line** (default): Press **Enter** to send immediately after typing your message
  598. - **Multiline**: Press **Enter** to add a new line, then press **Esc** followed by **Enter** to send the entire message when you're done. This allows for more complex messages with multiple paragraphs or code blocks.
  599. - **Ctrl+J** also inserts a new line in multiline mode as a reliable fallback across terminals
  600. Use `/input-mode` or `/im` during chat to open the interactive picker.
  601. > [!IMPORTANT]
  602. > Multiline send shortcuts can vary by terminal emulator and OS keyboard handling. This client relies on **Esc then Enter** as the portable submit shortcut in multiline mode. **Shift+Enter** and **Meta+Enter** may work in some terminals, but they are not guaranteed.
  603. ### Model Selection
  604. The model selection interface shows all available models in your Ollama installation:
  605. ![ollmcp model selection interface](https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/ollmpc-model-selection.jpg?raw=true)
  606. - Enter the **number** of the model you want to use
  607. - `s` or `save` - Save the model selection and return to chat
  608. - `q` or `quit` - Cancel the model selection and return to chat
  609. ### Advanced Model Configuration
  610. The `/model-config` (`/mc`) command opens the advanced model settings interface, allowing you to fine-tune how the model generates responses:
  611. ![ollmcp model configuration interface](https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/ollmcp-model-configuration.png?raw=true)
  612. #### System Prompt
  613. - **System Prompt**: Set the model's role and behavior to guide responses.
  614. #### Key Parameters
  615. - **Context Window (num_ctx)**: Set how much chat history the model uses. Balance with memory usage and performance.
  616. - **Keep Tokens**: Prevent important tokens from being dropped
  617. - **Max Tokens**: Limit response length (0 = auto)
  618. - **Seed**: Make outputs reproducible (set to -1 for random)
  619. - **Temperature**: Control randomness (0 = deterministic, higher = creative)
  620. - **Top K / Top P / Min P / Typical P**: Sampling controls for diversity
  621. - **Repeat Last N / Repeat Penalty**: Reduce repetition
  622. - **Presence/Frequency Penalty**: Encourage new topics, reduce repeats
  623. - **Stop Sequences**: Custom stopping points (up to 8)
  624. - **Batch Size (num_batch)**: Controls internal batching of requests; larger values can increase throughput but use more memory.
  625. #### Commands
  626. - Enter parameter numbers `1-15` to edit settings
  627. - Enter `sp` to edit the system prompt
  628. - Use `u1`, `u2`, etc. to unset parameters, or `uall` to reset all
  629. - `h`/`help`: Show parameter details and tips
  630. - `undo`: Revert changes
  631. - `s`/`save`: Apply changes
  632. - `q`/`quit`: Cancel
  633. #### Example Configurations
  634. - **Factual:** `temperature: 0.0-0.3`, `top_p: 0.1-0.5`, `seed: 42`
  635. - **Creative:** `temperature: 1.0+`, `top_p: 0.95`, `presence_penalty: 0.2`
  636. - **Reduce Repeats:** `repeat_penalty: 1.1-1.3`, `presence_penalty: 0.2`, `frequency_penalty: 0.3`
  637. - **Balanced:** `temperature: 0.7`, `top_p: 0.9`, `typical_p: 0.7`
  638. - **Reproducible:** `seed: 42`, `temperature: 0.0`
  639. - **Large Context:** `num_ctx: 8192` or higher for complex conversations requiring more context
  640. > [!TIP]
  641. > All parameters default to unset, letting Ollama use its own optimized values. Use `help` in the config menu for details and recommendations. Changes are saved with your configuration.
  642. ### Thinking Mode and Reasoning Effort
  643. Enable thinking mode with `/thinking-mode` (`/tm`) to activate extended reasoning on supported models (e.g., `qwen3`, `deepseek-r1`, Claude with extended thinking). Use `/show-thinking` (`/st`) to toggle whether the reasoning process is visible in the response.
  644. Use `/reasoning-effort` (`/re`) to control **how much reasoning effort** the model applies when thinking mode is on:
  645. | Level | Description |
  646. |-------|-------------|
  647. | `auto` | Provider's default effort (recommended for cloud) |
  648. | `minimal` | Fastest, least reasoning |
  649. | `low` | Light reasoning |
  650. | `medium` | Balanced — **default** |
  651. | `high` | More thorough reasoning |
  652. | `xhigh` | Maximum reasoning effort |
  653. > [!NOTE]
  654. > Some providers or models may ignore reasoning effort settings.
  655. ### Server Reloading for Development
  656. The `/reload-servers` command (`/rs`) is particularly useful during MCP server development. It allows you to reload all connected servers without restarting the entire client application.
  657. **Key Benefits:**
  658. - 🔄 **Hot Reload**: Instantly apply changes to your MCP server code
  659. - 🛠️ **Development Workflow**: Perfect for iterative development and testing
  660. - 📝 **Configuration Updates**: Automatically picks up changes in server JSON configs or Claude configs
  661. - 🎯 **State Preservation**: Maintains your tool enabled/disabled preferences across reloads
  662. - ⚡️ **Time Saving**: No need to restart the client and reconfigure everything
  663. **When to Use:**
  664. - After modifying your MCP server implementation
  665. - When you've updated server configurations in JSON files
  666. - After changing Claude's MCP configuration
  667. - During debugging to ensure you're testing the latest server version
  668. Simply type `/reload-servers` or `/rs` in the chat interface, and the client will:
  669. 1. Disconnect from all current MCP servers
  670. 2. Reconnect using the same parameters (servers added via `ollmcp mcp add`, server paths, config files, `--claude-desktop`)
  671. 3. Restore your previous tool enabled/disabled settings
  672. 4. Display the updated server and tool status
  673. This feature dramatically improves the development experience when building and testing MCP servers.
  674. ### Human-in-the-Loop (HIL) Tool Execution
  675. The Human-in-the-Loop feature provides an additional safety layer by allowing you to review and approve tool executions before they run. This is particularly useful for:
  676. - 🛡️ **Safety**: Review potentially destructive operations before execution
  677. - 🔍 **Learning**: Understand what tools the model wants to use and why
  678. - 🎯 **Control**: Selective execution of only the tools you approve
  679. - 🚫 **Prevention**: Stop unwanted tool calls from executing
  680. - 🔄 **Session Mode**: Auto-approve all tools for the current query session
  681. - 🛑 **Query Abort**: Abort entire query without saving to history
  682. #### HIL Confirmation Display
  683. When HIL is enabled, you'll see a confirmation prompt before each tool execution:
  684. **Example:**
  685. ![ollmcp HIL confirmation screenshot](https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/ollmcp-hil-feature.png?raw=true)
  686. #### HIL Confirmation Options
  687. When prompted, you can choose from the following options:
  688. - **y/yes**: Execute this specific tool call
  689. - **n/no**: Skip this tool call and continue with the query
  690. - **s/session**: Execute this and all subsequent tool calls for the current query without further prompts
  691. - **d/disable**: Permanently disable HIL confirmations (can be re-enabled with the `/hil` command)
  692. - **a/abort**: Abort the entire query immediately without saving to history
  693. > [!TIP]
  694. > The **session** option is particularly useful when the model needs to execute multiple tools in sequence. Instead of confirming each one individually, you can approve all tools for the current query session, then HIL will reset automatically for the next query.
  695. #### Human-in-the-Loop (HIL) Configuration
  696. - **Default State**: HIL confirmations are enabled by default for safety
  697. - **Toggle Command**: Use `/human-in-the-loop` or `/hil` to toggle on/off
  698. - **Persistent Settings**: HIL preference is saved with your configuration
  699. - **Quick Disable**: Choose "disable" during any confirmation to turn off permanently
  700. - **Session Auto-Approve**: Use "session" during confirmation to approve all tools for current query
  701. - **Query Abort**: Use "abort" during confirmation to immediately stop the query without saving
  702. - **Re-enable**: Use the `/hil` command anytime to turn confirmations back on
  703. **Benefits:**
  704. - **Enhanced Safety**: Prevent accidental or unwanted tool executions
  705. - **Awareness**: Understand what actions the model is attempting to perform
  706. - **Selective Control**: Choose which operations to allow on a case-by-case basis
  707. - **Flexible Workflow**: Session mode for efficient multi-tool queries, individual approval for sensitive operations
  708. - **Clean Abort**: Stop problematic queries immediately without polluting conversation history
  709. - **Peace of Mind**: Full visibility and control over automated actions
  710. ### Performance Metrics
  711. The Performance Metrics feature displays detailed model performance data after each query in a bordered panel. The metrics show duration timings, token counts, and generation rates directly from Ollama's response.
  712. **Displayed Metrics:**
  713. - `total duration`: Total time spent generating the complete response (seconds)
  714. - `load duration`: Time spent loading the model (milliseconds)
  715. - `prompt eval count`: Number of tokens in the input prompt
  716. - `prompt eval duration`: Time spent evaluating the input prompt (milliseconds)
  717. - `eval count`: Number of tokens generated in the response
  718. - `eval duration`: Time spent generating the response tokens (seconds)
  719. - `prompt eval rate`: Speed of input prompt processing (tokens/second)
  720. - `eval rate`: Speed of response token generation (tokens/second)
  721. **Example:**
  722. ![ollmcp ollama performance metrics screenshot](https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/ollmcp-ollama-performance-metrics.png?raw=true)
  723. #### Performance Metrics Configuration
  724. - **Default State**: Metrics are disabled by default for cleaner output
  725. - **Toggle Command**: Use `/show-metrics` or `/sm` to enable/disable metrics display
  726. - **Persistent Settings**: Metrics preference is saved with your configuration
  727. **Benefits:**
  728. - **Performance Monitoring**: Track model efficiency and response times
  729. - **Token Tracking**: Monitor actual token consumption for analysis
  730. - **Benchmarking**: Compare performance across different models
  731. > [!NOTE]
  732. > **Data Source**: All metrics come directly from Ollama's response, ensuring accuracy and reliability.
  733. ### History Management
  734. The History Management feature allows you to view, export, and import your conversation history. This is useful for:
  735. - 📜 **Full History View**: Review all conversations from your current session
  736. - 💾 **Export**: Save conversations to JSON files for backup or analysis
  737. - 📥 **Import**: Load previous conversation history to continue where you left off
  738. - 🔄 **Portability**: Share or transfer conversations between sessions
  739. #### History Commands
  740. **View Full History:**
  741. ```
  742. /full-history # or '/fh'
  743. ```
  744. Displays all conversation history from the current session in a formatted view, showing both queries and responses.
  745. **Export History:**
  746. ```
  747. /export-history # or '/eh'
  748. ```
  749. Exports your current chat history to a JSON file. You can specify a custom filename or use the default timestamp-based name (e.g., `ollmcp_chat_history_2026-01-05_143022.json`). Files are saved to `~/.config/ollmcp/history/` directory. The command includes file overwrite protection.
  750. **Import History:**
  751. ```
  752. /import-history # or '/ih'
  753. ```
  754. Imports a previously exported chat history from a JSON file. The command validates the JSON structure to ensure compatibility. Imported history is added to your current conversation context.
  755. **History Storage:**
  756. - Export location: `~/.config/ollmcp/history/`
  757. - Default filename format: `ollmcp_chat_history_YYYY-MM-DD_HHMMSS.json`
  758. - JSON format includes both queries and responses with proper structure validation
  759. **Benefits:**
  760. - **Session Continuity**: Resume conversations across different sessions
  761. - **Backup**: Keep records of important conversations
  762. - **Analysis**: Export history for external analysis or review
  763. - **Sharing**: Share conversation context with team members
  764. - **Testing**: Import test conversations for development and debugging
  765. > [!TIP]
  766. > When exporting, if you don't provide a filename, the system automatically generates a timestamped filename to prevent accidental overwrites.
  767. ## Autocomplete and Prompt Features
  768. ### Typer Shell Autocompletion
  769. - The CLI supports shell autocompletion for all options and arguments via Typer
  770. - To enable, run `ollmcp --install-completion` and follow the instructions for your shell
  771. - Enjoy tab-completion for all grouped and general options
  772. ### FZF-style Autocomplete
  773. - Slash-namespace autocomplete for commands and prompts (`/`)
  774. - Command descriptions shown in the menu
  775. - Case-insensitive matching for convenience
  776. - Centralized command list for consistency
  777. - Plain text query typing is intentionally free of action autocomplete noise
  778. ### MCP Prompts Autocomplete
  779. - Type `/` to trigger prompt autocomplete
  780. - Fuzzy matching on prompt names and descriptions
  781. - Supports qualified prompt references like `/server:prompt_name`
  782. - Shows prompt descriptions in the menu
  783. - Prompt arguments are collected during prompt invocation (not shown in autocomplete rows)
  784. - Terminal-width-aware description truncation
  785. ### Contextual Prompt
  786. The chat prompt now gives you clear, contextual information at a glance:
  787. - **Model**: Shows the current Ollama model in use
  788. - **Thinking Mode**: Indicates if "thinking mode" is active (for supported models)
  789. - **Tools**: Displays the number of enabled tools
  790. **Example prompt:**
  791. ```
  792. qwen3/show-thinking/12-tools❯
  793. ```
  794. - `qwen3` Model name
  795. - `/show-thinking` Thinking mode indicator (if enabled, otherwise `/thinking` or omitted)
  796. - `/12-tools` Number of tools enabled (or `/1-tool` for singular)
  797. - `❯` Prompt symbol
  798. This makes it easy to see your current context before entering a query.
  799. > [!TIP]
  800. > Type `/` after the prompt symbol to see autocomplete suggestions for available MCP prompts.
  801. ## Configuration Management
  802. > [!TIP]
  803. > Running `ollmcp` with no flags automatically loads the default configuration from `~/.config/ollmcp/config.json` if it exists.
  804. The client saves and loads your preferences between sessions:
  805. - When using `/save-config`, you can provide a name for the configuration or use the default
  806. - Configurations are stored in the `~/.config/ollmcp/` directory
  807. - The default configuration is saved as `~/.config/ollmcp/config.json`
  808. - Named configurations are saved as `~/.config/ollmcp/{name}.json`
  809. ### Per-provider profiles
  810. Connection settings are stored **per provider**, so switching providers never reuses another provider's model, host, or API key. Each provider keeps its own:
  811. - **Model** selection
  812. - **Host** / API base URL
  813. - **API key**
  814. The configuration also records a `defaultProvider`. When you run `ollmcp` with no `--provider` flag, it loads that provider's profile; a fresh install starts on `ollama`. Each time you `/save-config`, the provider you're currently using *becomes the new default*, so running plain `ollmcp` resumes wherever you left off. Pass `--provider <name>` at any time to switch to (and load) a different provider's profile, and `--model` / `--host` / `--api-key` override the saved values for that run.
  815. > [!NOTE]
  816. > Only a key passed with `--api-key` is stored, in plain text, in `~/.config/ollmcp/config.json`. Keys provided through the `$OLLMCP_API_KEY` environment variable or a provider's native environment variable (for example `OPENROUTER_API_KEY`) are **never** written to disk, use one of those if you don't want your key persisted.
  817. The following settings are **shared** across all providers:
  818. - Advanced model parameters (system prompt, temperature, sampling settings, etc.)
  819. - Enabled/disabled status of all tools
  820. - Context retention settings
  821. - Thinking mode settings
  822. - Answer display mode preference
  823. - Tool execution display preferences
  824. - Performance metrics display preferences
  825. - Human-in-the-Loop confirmation settings
  826. **Example `~/.config/ollmcp/config.json`:**
  827. ```json
  828. {
  829. "defaultProvider": "openai",
  830. "providers": {
  831. "ollama": { "host": "http://localhost:11434", "model": "qwen3:1.7b", "apiKey": "" },
  832. "openai": { "host": "", "model": "gpt-5.5", "apiKey": "sk-..." }
  833. },
  834. "enabledTools": {},
  835. "modelConfig": {},
  836. "...": "shared settings"
  837. }
  838. ```
  839. > [!TIP]
  840. > Older flat config files (with top-level `host`/`model`/`provider`/`apiKey`) are migrated automatically the first time you run this version, and rewritten in the per-provider format on your next `/save-config`.
  841. ## Server Configuration Format
  842. The JSON configuration file supports STDIO, SSE, and Streamable HTTP server types (MCP 1.10.1):
  843. ```json
  844. {
  845. "mcpServers": {
  846. "stdio-server": {
  847. "command": "command-to-run",
  848. "args": ["arg1", "arg2", "..."],
  849. "env": {
  850. "ENV_VAR1": "value1",
  851. "ENV_VAR2": "value2"
  852. },
  853. "disabled": false
  854. },
  855. "sse-server": {
  856. "type": "sse",
  857. "url": "http://localhost:8000/sse",
  858. "headers": {
  859. "Authorization": "Bearer your-token-here"
  860. },
  861. "disabled": true
  862. },
  863. "http-server": {
  864. "type": "streamable_http",
  865. "url": "http://localhost:8000/mcp",
  866. "headers": {
  867. "X-API-Key": "your-api-key-here"
  868. },
  869. "disabled": false
  870. }
  871. }
  872. }
  873. ```
  874. > [!NOTE]
  875. > **MCP 1.10.1 Transport Support**: The client now supports the latest Streamable HTTP transport with improved performance and reliability. If you specify a URL without a type, the client will default to using Streamable HTTP transport.
  876. ### Tips: where to put MCP server configs and a working example
  877. A common point of confusion is where to store MCP server configuration files and how the TUI's save/load feature is used. Here's a short, practical guide that has helped other users:
  878. - The TUI's `/save-config` / `/load-config` (or `/sc` / `/lc`) commands are intended to save *TUI preferences* like which tools you enabled, your selected model, thinking mode, display mode, and other client-side settings. They are not required to register MCP server connections with the client.
  879. - For MCP server JSON files (the `mcpServers` object shown above) we recommend keeping them outside the TUI config directory or in a clear subfolder, for example:
  880. ```
  881. ~/.config/ollmcp/mcp-servers/config.json
  882. ```
  883. You can then point `ollmcp` at that file at startup with `-j` / `--servers-json`.
  884. > [!IMPORTANT]
  885. > For HTTP-based MCP servers, `"type": "http"`, `"streamable-http"`, and `"streamable_http"` are all accepted and treated the same way. Also check the [Common MCP endpoint paths](#common-mcp-endpoint-paths) section below for typical endpoints.
  886. Here a minimal working example let's say this is your `~/.config/ollmcp/mcp-servers/config.json`:
  887. ```json
  888. {
  889. "mcpServers": {
  890. "github": {
  891. "type": "streamable_http",
  892. "url": "https://api.githubcopilot.com/mcp/",
  893. "headers": {
  894. "Authorization": "Bearer mytoken"
  895. }
  896. }
  897. }
  898. }
  899. ```
  900. > [!TIP]
  901. > When using GitHub MCP server, make sure to replace `"mytoken"` with your actual GitHub API token.
  902. With that file in place you can connect using:
  903. ```
  904. ollmcp -j ~/.config/ollmcp/mcp-servers/config.json
  905. ```
  906. Here you can find a GitHub issue related to this common pitfall: https://github.com/jonigl/mcp-client-for-ollama/issues/112#issuecomment-3446569030
  907. #### Demo
  908. A short demo (asciicast) that should help anyone reproduce the working setup quickly. This example uses an [MCP server example with streamable HTTP protocol](https://github.com/jonigl/mcp-server-with-streamable-http-example) usage:
  909. [![asciicast](https://asciinema.org/a/751387.svg)](https://asciinema.org/a/751387)
  910. #### Common MCP endpoint paths
  911. Streamable HTTP MCP servers typically expose the MCP endpoint at `/mcp` (e.g., `https://host/mcp`), while SSE servers commonly use `/sse` (e.g., `https://host/sse`). Below is an excerpt from the MCP specification (2025-06-18):
  912. > The server MUST provide a single HTTP endpoint path (hereafter referred to as the MCP endpoint) that supports both POST and GET methods. For example, this could be a URL like https://example.com/mcp.
  913. You can find more details in the [MCP specification version 2025-06-18 - Transports](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports).
  914. > [!NOTE]
  915. > HTTPS certificates for remote MCP servers are verified against your **operating system trust store**, not the `certifi` bundle. If an MCP server sits behind a private or corporate CA, or you run ollmcp in a minimal container with no system CA store, the TLS handshake fails with an error that does not mention ollmcp. Point `SSL_CERT_FILE` (a bundle file) or `SSL_CERT_DIR` (a directory) at your CA to fix it:
  916. >
  917. > ```bash
  918. > export SSL_CERT_FILE=/path/to/corporate-ca.pem
  919. > ```
  920. ## Compatible Models
  921. The following Ollama models work well with tool use:
  922. - gemma4
  923. - qwen3.5
  924. - lfm2.5-thinking
  925. - llama3.2
  926. - mistral
  927. For a complete list of Ollama models with tool use capabilities, visit the [official Ollama models page](https://ollama.com/search?c=tools).
  928. For models that can also process images returned by tools, see the [Ollama vision models page](https://ollama.com/search?c=vision).
  929. ### Ollama Cloud Models
  930. MCP Client for Ollama now supports [Ollama Cloud models](https://github.com/ollama/ollama/blob/main/docs/cloud.md), allowing you to use powerful cloud-hosted models with tool calling capabilities while leveraging your local MCP tools. Cloud models can run without a powerful local GPU, making it possible to access larger models that wouldn't fit on a personal computer.
  931. **Supported Ollama Cloud models include for example:**
  932. - `gpt-oss:20b-cloud`
  933. - `gpt-oss:120b-cloud`
  934. - `deepseek-v3.1:671b-cloud`
  935. - `qwen3-coder:480b-cloud`
  936. **To use Ollama Cloud models with this client:**
  937. 1. First, pull the cloud model:
  938. ```bash
  939. ollama pull gpt-oss:120b-cloud
  940. ```
  941. 2. Run the client with your chosen cloud model:
  942. ```bash
  943. ollmcp --model gpt-oss:120b-cloud
  944. ```
  945. > [!NOTE]
  946. > The model `deepseek-v3.1:671b-cloud` only supports tool use when thinking mode is turned off. You can toggle thinking mode in `ollmcp` by typing either `/thinking-mode` or `/tm`.
  947. For more information about Ollama Cloud, visit the [Ollama Cloud documentation](https://docs.ollama.com/cloud).
  948. ## Sponsors
  949. This project is supported by:
  950. <p align="center">
  951. <a href="https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=mcp-client-for-ollama">
  952. <picture>
  953. <source media="(prefers-color-scheme: dark)" srcset="https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/atlascloud-logo-dark.png?raw=true">
  954. <img src="https://github.com/jonigl/mcp-client-for-ollama/blob/main/misc/atlascloud-logo.png?raw=true" alt="Atlas Cloud" height="36">
  955. </picture>
  956. </a>
  957. </p>
  958. ### Atlas Cloud
  959. [Atlas Cloud](https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=mcp-client-for-ollama) is a full-modal AI inference platform — a single AI API with unified access to 300+ curated models for video, image, and LLM workloads.
  960. It works with ollmcp out of the box:
  961. ```bash
  962. ollmcp --provider atlascloud --api-key YOUR_ATLASCLOUD_API_KEY
  963. # or export the key once and skip the flag:
  964. export ATLASCLOUD_API_KEY=YOUR_ATLASCLOUD_API_KEY
  965. ollmcp --provider atlascloud
  966. ```
  967. Streaming and tool calling are fully supported, and you can browse and pick any of Atlas Cloud's models with the interactive `/model` command.
  968. ### Become a sponsor
  969. Want to see your logo here? Support the project via [GitHub Sponsors](https://github.com/sponsors/jonigl) ❤️
  970. ## Where Can I Find More MCP Servers?
  971. You can explore a collection of MCP servers in the official [MCP Servers repository](https://github.com/modelcontextprotocol/servers).
  972. This repository contains reference implementations for the Model Context Protocol, community-built servers, and additional resources to enhance your LLM tool capabilities.
  973. ## Related Projects
  974. - **[Ollama MCP Bridge](https://github.com/jonigl/ollama-mcp-bridge)** - A Python API layer that sits in front of Ollama, automatically adding tools from multiple MCP servers to every chat request. This project provides a transparent proxy solution that pre-loads all MCP servers at startup and seamlessly integrates their tools into the Ollama API.
  975. - **[MCP Server with Streamable HTTP Example](https://github.com/jonigl/mcp-server-with-streamable-http-example)** - An example MCP server demonstrating the usage of the streamable HTTP protocol.
  976. ## Security
  977. MCP servers you connect are trusted by you, and their tool/resource responses are treated as untrusted content that reaches the model. See [SECURITY.md](SECURITY.md) for the trust model, how indirect prompt injection is handled, and how to report a vulnerability.
  978. ## License
  979. This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
  980. ## Acknowledgments
  981. - [Ollama](https://ollama.com/) for the local LLM runtime
  982. - [Model Context Protocol](https://modelcontextprotocol.io/) for the specification and examples
  983. - [any-llm](https://github.com/mozilla-ai/any-llm/) for the single interface to multiple LLM providers
  984. - [Rich](https://rich.readthedocs.io/) for the terminal user interface
  985. - [Typer](https://typer.tiangolo.com/) for the modern CLI experience
  986. - [Prompt Toolkit](https://python-prompt-toolkit.readthedocs.io/) for the interactive command line interface
  987. - [uv](https://github.com/astral-sh/uv) for the lightning-fast Python package manager and virtual environment management
  988. - [Asciinema](https://asciinema.org/) for the demo recording
  989. ---
  990. Made with ❤️ by [jonigl](https://github.com/jonigl)

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.