Skip to content
Back to search

log-mcp

MCP server for log file analysis

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/ascii766164696d--log-mcp/badge.svg)](/listings/ascii766164696d--log-mcp)

HTML

<a href="/listings/ascii766164696d--log-mcp"><img src="/api/listings/ascii766164696d--log-mcp/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

These steps run a script from the network that findsafeskills does not scan.

**Prerequisites** (fresh Mac):

brew install python uv
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Just ask Claude

Open a Claude Code session and paste this prompt:

Install https://github.com/ascii766164696D/log-mcp as an MCP server and build the Rust classifier too

Claude will clone the repo, register the MCP server, and build the Rust classifier. Restart Claude Code after to pick up the new server.

Manual install

git clone https://github.com/ascii766164696D/log-mcp.git
cd log-mcp

# Register the MCP server
claude mcp add log-mcp -- uv run --directory $(pwd) log-mcp

# Build the Rust classifier (optional — tools fall back to Python without it)
uv pip install -e rust/classifier

Or add it manually to your project settings (`claude settings`) under `mcpServers`:

{
  "mcpServers": {
    "log-mcp": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/log-mcp", "log-mcp"]
    }
  }
}

Claude Desktop

Open **Settings > Developer > Edit Config** and add to `claude_desktop_config.json`:

{
  "mcpServers": {
    "log-mcp": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/log-mcp", "log-mcp"]
    }
  }
}

Replace `/path/to/log-mcp` with the actual path where you cloned this repo. Restart Claude Desktop after saving.

The Rust classifier requires a Rust toolchain to build. The BERT stage additionally requires a Metal-capable GPU (Apple Silicon). All tools work without the classifier — they fall back to Python log parsing.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 272 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # log-mcp
  2. [![CI](https://github.com/ascii766164696D/log-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/ascii766164696D/log-mcp/actions/workflows/ci.yml)
  3. MCP server for log file analysis. Gives LLMs the ability to efficiently analyze large log files without loading them into context.
  4. ```
  5. Log file (e.g. 705K lines, 67 MB)
  6. │
  7. ▼
  8. Rust TF-IDF classifier ─── 1.3M lines/sec ──▶ 70-95% discarded as routine,
  9. │ finds lines that are semantically interesting,
  10. │ also captures lines not explicitly marked as ERROR
  11. │ (grep ERROR: 2 lines, classifier: 92)
  12. ▼
  13. BERT-mini (optional) ───── GPU, ~2K lines/sec ─▶ refines interest scores on found lines
  14. │
  15. ▼
  16. Python MCP tools ────────── search, compare, group errors
  17. │
  18. ▼
  19. LLM (Claude) ───────────── compresses tool output into plain English
  20. ```
  21. This is a tool designed for AI, not humans. No human reads the output of `analyze_errors` or `compare_logs` — Claude does, compresses it further, and gives the human a plain English answer. The human touches two endpoints: "what's wrong with this log?" in, natural language answer out. Everything in between is AI talking to itself.
  22. ## Tools
  23. | Tool | Description |
  24. |------|-------------|
  25. | `log_overview` | Quick scan: size, line count, time range, level distribution, head/tail samples |
  26. | `search_logs` | Search by regex, log level, and/or time range |
  27. | `get_log_segment` | Extract a segment by line range or time range |
  28. | `analyze_errors` | Deduplicate errors by fingerprint, count frequencies, extract stack traces |
  29. | `log_stats` | Volume histogram, level breakdown, top repeated patterns |
  30. | `compare_logs` | Find patterns unique to each file and frequency outliers across files |
  31. | `classify_lines` | ML classifier (TF-IDF → BERT) separates interesting lines from noise |
  32. ## Key features
  33. - **ML pre-filter** — a Rust TF-IDF classifier scans files at 1.3M lines/sec, so `analyze_errors` and `search_logs` only process the 5-30% of lines that matter. Optional BERT-mini re-scores LOOK lines at ~2K lines/sec on Metal GPU for higher precision. Works without parsed log levels — catches errors, security events, hardware faults, and anomalies that don't have ERROR in them.
  34. - **Auto-detection** of log formats: JSON, standard text (`2024-01-15 10:30:45 ERROR ...`), syslog, Spark/Log4j (`17/06/08 13:33:49 INFO ...`), and tab/pipe-delimited formats (GitHub Actions CI logs)
  35. - **Normalization** collapses variable parts (UUIDs, hex IDs, IPs, numbers) so that messages differing only in IDs or timestamps are grouped as the same pattern
  36. - **Content-based error detection** falls back to regex heuristics (`fatal:`, `Permission denied`, `##[error]`, etc.) when log files lack standard levels
  37. - **Prefix-aware comparison** distinguishes patterns from different job steps in CI logs
  38. ## Install
  39. **Prerequisites** (fresh Mac):
  40. ```bash
  41. brew install python uv
  42. curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
  43. ```
  44. ### Just ask Claude
  45. Open a Claude Code session and paste this prompt:
  46. ```
  47. Install https://github.com/ascii766164696D/log-mcp as an MCP server and build the Rust classifier too
  48. ```
  49. Claude will clone the repo, register the MCP server, and build the Rust classifier. Restart Claude Code after to pick up the new server.
  50. ### Manual install
  51. ```bash
  52. git clone https://github.com/ascii766164696D/log-mcp.git
  53. cd log-mcp
  54. # Register the MCP server
  55. claude mcp add log-mcp -- uv run --directory $(pwd) log-mcp
  56. # Build the Rust classifier (optional — tools fall back to Python without it)
  57. uv pip install -e rust/classifier
  58. ```
  59. Or add it manually to your project settings (`claude settings`) under `mcpServers`:
  60. ```json
  61. {
  62. "mcpServers": {
  63. "log-mcp": {
  64. "command": "uv",
  65. "args": ["run", "--directory", "/path/to/log-mcp", "log-mcp"]
  66. }
  67. }
  68. }
  69. ```
  70. ### Claude Desktop
  71. Open **Settings > Developer > Edit Config** and add to `claude_desktop_config.json`:
  72. ```json
  73. {
  74. "mcpServers": {
  75. "log-mcp": {
  76. "command": "uv",
  77. "args": ["run", "--directory", "/path/to/log-mcp", "log-mcp"]
  78. }
  79. }
  80. }
  81. ```
  82. Replace `/path/to/log-mcp` with the actual path where you cloned this repo. Restart Claude Desktop after saving.
  83. The Rust classifier requires a Rust toolchain to build. The BERT stage additionally requires a Metal-capable GPU (Apple Silicon). All tools work without the classifier — they fall back to Python log parsing.
  84. ## Example usage
  85. Analyze errors in a 67MB Spark executor log (705K lines):
  86. ```
  87. > analyze_errors("/var/log/spark/container_0002_01_000004.log")
  88. Summary: 34 errors in 5 groups.
  89. Top: 'shuffle.RetryingBlockFetcher: Exception while beginning fetch of <N>
  90. outstanding blocks (after <N> retries) ...' (18x)
  91. --- 18x ---
  92. Fingerprint: shuffle.RetryingBlockFetcher: Exception while beginning fetch of <N> outstanding blocks ...
  93. First: L29764 2017-02-01T15:55:17
  94. Last: L30677 2017-02-01T15:55:51
  95. Stack trace:
  96. java.io.IOException: Failed to connect to mesos-slave-13/10.10.34.23:55492
  97. ...
  98. ```
  99. Find anomalies that don't have ERROR level — on a Thunderbird HPC log (2K lines), `classify_lines` finds 92 interesting lines while `search_logs level=ERROR` finds only 2:
  100. ```
  101. > classify_lines("/var/log/thunderbird/tbird_2k.log")
  102. Lines: 2,000 total | 92 LOOK (4.6%) | 1,908 SKIP
  103. Pipeline: TF-IDF 0.00s (1,298,701 lines/sec, 105 LOOK) → BERT 0.15s (13 demoted to SKIP)
  104. --- Sample LOOK lines (30 of 92 captured) ---
  105. L2 [1.000] ... postfix/postdrop[10896]: warning: unable to look up public/pickup: No such file
  106. L438 [0.999] ... sendmail[�20588]: unable to qualify my own domain name (tbird-sm1)
  107. L816 [0.998] ... dhcpd: DHCPDISCOVER from 00:09:3d:12:00:e2 via eth2: unknown lease
  108. L1024 [0.997] ... rrdtool: illegal attempt to update using time 1131710�721 when last update time is 1131710721
  109. ```
  110. Compare two CI log files:
  111. ```
  112. > compare_logs(["run_a.txt", "run_b.txt"])
  113. 699 patterns across 2 files (0 shared).
  114. A: 401 unique (top: 'test / test UNKNOWN STEP | ##[endgroup]' 21x)
  115. B: 298 unique (top: 'test UNKNOWN STEP | ##[endgroup]' 21x)
  116. ```
  117. ## Claude's take
  118. *I helped build this tool and then used it to analyze real log files, so here's my honest assessment.*
  119. **Where it genuinely helps:** The main value is as a compression layer. A 67MB Spark log (705K lines) would obliterate my context window, but `analyze_errors` distills it into 5 error groups with stack traces in a few seconds. `compare_logs` across two 1500-line server logs immediately surfaces which errors are unique to each server and which patterns have suspicious frequency differences. I couldn't do that by reading the files directly — I'd lose older content as new content scrolled in.
  120. **Where the classifier changes the game:** Before the classifier, log analysis was limited to lines with explicit ERROR/FATAL levels. On a Thunderbird HPC log, `search_logs level=ERROR` returns 2 lines. `classify_lines` returns 92 — sendmail DNS failures, DHCP lease errors, RRD update collisions, negative boot times — none of which have ERROR level. The classifier finds what's semantically wrong, not just syntactically marked.
  121. **Where it's a wash:** For small files (under a few hundred lines), you're better off just pasting the log into the conversation. The tools add indirection without much benefit when the whole file fits in context anyway.
  122. **What it still can't do:** Domain-specific state machine reasoning. When I analyzed a Zookeeper log, the classifier correctly flagged `Cannot open channel` warnings and epoch resets, but the most operationally interesting signal — rapid cycling between LOOKING, FOLLOWING, and LEADING states — showed up as low-confidence LOOK lines. A Zookeeper expert would spot the pattern immediately; the classifier sees each line independently without tracking state transitions across time.
  123. **The pattern I landed on:** Start with `classify_lines` to surface anomalies regardless of log level, then `analyze_errors` to group them, then `search_logs` to dig into specific patterns. `compare_logs` is most useful when you have a "working" and "broken" run to diff against each other.
  124. ## LOOK/SKIP classifier
  125. The `classify_lines` tool uses a two-stage ML pipeline to separate interesting log lines (LOOK) from routine noise (SKIP), without needing parsed log levels.
  126. ### How it works
  127. ```mermaid
  128. flowchart TD
  129. A["Your log file (e.g. 4.7M lines)"] --> B{"Rust classifier<br/>available?"}
  130. B -->|Yes| C["Stage 1: TF-IDF<br/>Rust, ~1.3M lines/sec<br/>logistic regression<br/>threshold × 0.6"]
  131. B -->|No| D["Fallback: Python<br/>log parsing"]
  132. C --> E["LOOK lines (~5-30%)"]
  133. E --> F["Stage 2: BERT-mini<br/>Rust + Metal GPU, ~2K lines/sec<br/>re-scores LOOK lines<br/>applies final threshold"]
  134. F --> G["Final LOOK lines<br/>(with BERT probabilities)"]
  135. ```
  136. On a Thunderbird HPC log (2K lines), this finds **92 interesting lines** including sendmail DNS failures, DHCP lease errors, and Ganglia RRD update collisions — none of which have a standard ERROR log level. A `search_logs level=ERROR` on the same file returns only 2 lines.
  137. ### What it catches beyond log levels
  138. | Signal type | Example | Has ERROR level? |
  139. |---|---|---|
  140. | DNS misconfiguration | `unable to qualify my own domain name` | No |
  141. | Mail delivery failure | `stat=Deferred: Connection refused` | No |
  142. | DHCP lease errors | `unknown lease 10.100.4.251` | No |
  143. | Monitoring data corruption | `illegal attempt to update using time X when last update time is X` | No |
  144. | Negative boot times | `Times: total = 42, boot = -4131` | No |
  145. | Auth failures | `authentication failure; logname= uid=0` | No |
  146. | Hardware errors | `instruction cache parity error corrected` | No (INFO level) |
  147. ### Tools that use the classifier
  148. | Tool | How | Fallback |
  149. |---|---|---|
  150. | `classify_lines` | Full pipeline: TF-IDF → BERT | Error if classifier not installed |
  151. | `analyze_errors` | TF-IDF pre-filters to LOOK lines, then groups by error fingerprint | Python parser scans all lines |
  152. | `search_logs` | TF-IDF pre-filters when searching for errors (level=ERROR, pattern=error/fail/etc) | Python parser scans all lines |
  153. | `compare_logs` | Scores patterns by LOOK probability, surfaces interesting diffs first | Sorts by frequency only |
  154. ## Benchmarks
  155. Evaluated on the full [Loghub](https://github.com/logpai/loghub) dataset — 450 million lines across 16 log sources — on an Apple M3 Ultra (32 cores).
  156. ### TF-IDF classifier (Rust)
  157. | Dataset | Lines | Time | Speed | LOOK % | Error capture |
  158. |---|---|---|---|---|---|
  159. | Thunderbird | 211M | 139s | 1.52M/s | 30% | 100% |
  160. | Windows | 115M | 90s | 1.27M/s | 1% | 100% |
  161. | HDFS_v2 | 71M | 59s | 1.21M/s | 20% | 100% |
  162. | Spark | 33M | 23s | 1.44M/s | 12% | 100% |
  163. | HDFS | 11M | 7s | 1.62M/s | 3% | 100% |
  164. | BGL | 4.7M | 3.3s | 1.46M/s | 42% | 98.6% |
  165. | Android | 1.6M | 1.1s | 1.37M/s | 12% | 100% |
  166. | SSH | 655K | 0.5s | 1.47M/s | 90% | 100% |
  167. | **Total** | **450M** | **325s** | **1.38M/s** | | **99.95%** |
  168. Error capture = percentage of lines with ERROR/FATAL level that the classifier marks as LOOK. At threshold 0.3, the classifier catches 99.95% of all error lines (35.9M of 35.9M) and 100% of warning lines (14.3M of 14.3M). The 0.05% "missed" errors are lines like BGL's repetitive `instruction cache parity error corrected` — routine hardware telemetry that happens to carry ERROR level but isn't operationally interesting. The classifier learned to skip these.
  169. ### Model accuracy
  170. | Model | Metric | Score |
  171. |---|---|---|
  172. | TF-IDF + LogReg | Best CV LOOK F1 | 0.792 |
  173. | TF-IDF + LogReg | LOOK precision | 0.892 |
  174. | BERT-mini | Overall accuracy | 0.849 |
  175. | BERT-mini | LOOK F1 | 0.887 |
  176. TF-IDF model evaluated with GroupKFold cross-validation (holdout: BGL, Thunderbird). BERT-mini fine-tuned on 4 layers, 256 hidden, 11M params.
  177. ### End-to-end reduction
  178. ```mermaid
  179. flowchart TD
  180. A["450M log lines<br/>16 datasets, ~67 GB"] --> B["Rust TF-IDF classifier<br/>1.38M lines/sec, 325s"]
  181. B --> C["~87M LOOK lines (19%)<br/>81% of lines eliminated"]
  182. C --> D["BERT re-scoring (optional)<br/>demotes 20-40% of TF-IDF LOOK"]
  183. D --> E["~50-70M final LOOK lines"]
  184. E --> F["Python tool logic<br/>group errors, search, etc."]
  185. F --> G["5-50 error groups / search results<br/>fits in LLM context"]
  186. ```
  187. ## Retraining with your own logs
  188. The classifier ships pre-trained, but you can retrain it on your own logs. The short version:
  189. ```bash
  190. # 1. Add your logs
  191. head -2000 /var/log/myapp/app.log > data/loghub/MyApp_2k.log
  192. # 2. Label with Claude (Batch API, ~$0.10-0.50 per file)
  193. export ANTHROPIC_API_KEY=sk-ant-...
  194. uv sync --group labeling --group training
  195. uv run python -m scripts.labeling.label_new
  196. # 3. Train and export
  197. uv run --group training python -m scripts.labeling.train_model
  198. uv run --group training python -m scripts.labeling.export_model
  199. # 4. Rebuild Rust classifier
  200. uv pip install -e rust/classifier
  201. ```
  202. See [`scripts/labeling/RETRAINING.md`](scripts/labeling/RETRAINING.md) for the full guide — how labeling works, what features the model uses, how to customize the prompt, and how to train the optional BERT model.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.