Skip to content
Back to search

sentinel-sre

Autonomous on-call SRE agent built from first principles without orchestration frameworks. Uses a custom MCP server, E2B Firecracker sandboxes, and a stateful human-in-the-loop control loop to diagnose production bottlenecks and execute verified runbooks.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/u-anjum--sentinel-sre/badge.svg)](/listings/u-anjum--sentinel-sre)

HTML

<a href="/listings/u-anjum--sentinel-sre"><img src="/api/listings/u-anjum--sentinel-sre/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

1. Install Dependencies & Configure Environment

uv sync
cp .env.example .env

Edit `.env` and add your `GEMINI_API_KEY` and `E2B_API_KEY`.

2. Start the Target Testbed

cd testbed
docker compose up -d --build
cd ..

See [`testbed/README.md`](testbed/README.md) for testbed endpoints and teardown commands.

3. Run the Control Plane & HITL Workflow

**Terminal 1 — Start the FastAPI Control Plane:**

uv run uvicorn api.main:app --port 8000

**Terminal 2 — Inject a database row-lock bottleneck:**

uv run python testbed/chaos/inject_db_lock.py

**Terminal 3 — Trigger an alert webhook, inspect triage, and approve remediation:**

# 1. Trigger alert webhook
curl -X POST "http://localhost:8000/webhooks/alert" \
  -H "Content-Type: application/json" \
  -d '{"alert_name":"HighOrderFailureRate","service":"target-api","symptom":"POST /orders returning 500 errors"}'

# 2. Inspect structured IncidentReport (replace <incident_id>)
curl "http://localhost:8000/incidents/<incident_id>"

# 3. Approve remediation (terminates blocking Postgres PID via MCP)
curl -X POST "http://localhost:8000/incidents/<incident_id>/approve" \
  -H "Content-Type: application/json" \
  -d '{"approved":true,"operator":"oncall-engineer","comment":"Verified blocker PID"}'

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 133 lines
  1. # sentinel-sre
  2. Autonomous Site Reliability Engineering (SRE) triage and remediation agent built from first principles—combining a custom Model Context Protocol (MCP) telemetry server, isolated E2B Firecracker sandboxes for log analysis, a raw Python control loop powered by Gemini, and a FastAPI Human-in-the-Loop (HITL) control plane.
  3. ## Architecture
  4. ```
  5. [Monitoring / Alertmanager]
  6. │ POST /webhooks/alert
  7. ▼
  8. ┌───────────────────────────────────────────────────────────┐
  9. │ Control Plane API (src/api) & SQLite Store (src/db) │
  10. │ • Persists incident lifecycle (TRIAGING → AWAITING_APPROVAL)│
  11. │ • Gates destructive remediation behind /approve │
  12. └───────────────────────────┬───────────────────────────────┘
  13. │ async background triage
  14. ▼
  15. ┌───────────────────────────────────────────────────────────┐
  16. │ Agent Core (src/agent_core) │
  17. │ • Pure-Python async control loop (Gemini + Pydantic v2) │
  18. │ • Enforces read-only triage & structured IncidentReport │
  19. └───────────────┬───────────────────────────┬───────────────┘
  20. │ stdio (MCP 2.x) │ HTTPS
  21. ▼ ▼
  22. ┌───────────────────────────────┐ ┌───────────────────────┐
  23. │ Telemetry MCP Server │ │ E2B MicroVM Sandbox │
  24. │ (src/mcp_servers) │ │ (/home/user) │
  25. │ • inspect_db_health │ │ • Receives streamed │
  26. │ • fetch_logs_to_sandbox ─────┼──▶│ logs.jsonl │
  27. │ • execute_terminate_pid* │ │ • Runs diagnostic │
  28. └───────────────┬───────────────┘ │ Python scripts │
  29. │ SQL └───────────────────────┘
  30. ▼
  31. ┌───────────────────────────────┐
  32. │ Target Testbed (testbed/) │
  33. │ • FastAPI + PostgreSQL 16 │
  34. │ • Reproducible chaos scripts │
  35. └───────────────────────────────┘
  36. *Executed only after explicit human sign-off via /incidents/{id}/approve.
  37. ```
  38. ## Prerequisites
  39. * **Python 3.12+** & [`uv`](https://docs.astral.sh/uv/)
  40. * **Docker Desktop** (Linux containers)
  41. * **API Keys:** `GEMINI_API_KEY` and `E2B_API_KEY`
  42. ## Quickstart
  43. ### 1. Install Dependencies & Configure Environment
  44. ```bash
  45. uv sync
  46. cp .env.example .env
  47. ```
  48. Edit `.env` and add your `GEMINI_API_KEY` and `E2B_API_KEY`.
  49. ### 2. Start the Target Testbed
  50. ```bash
  51. cd testbed
  52. docker compose up -d --build
  53. cd ..
  54. ```
  55. See [`testbed/README.md`](testbed/README.md) for testbed endpoints and teardown commands.
  56. ### 3. Run the Control Plane & HITL Workflow
  57. **Terminal 1 — Start the FastAPI Control Plane:**
  58. ```bash
  59. uv run uvicorn api.main:app --port 8000
  60. ```
  61. **Terminal 2 — Inject a database row-lock bottleneck:**
  62. ```bash
  63. uv run python testbed/chaos/inject_db_lock.py
  64. ```
  65. **Terminal 3 — Trigger an alert webhook, inspect triage, and approve remediation:**
  66. ```bash
  67. # 1. Trigger alert webhook
  68. curl -X POST "http://localhost:8000/webhooks/alert" \
  69. -H "Content-Type: application/json" \
  70. -d '{"alert_name":"HighOrderFailureRate","service":"target-api","symptom":"POST /orders returning 500 errors"}'
  71. # 2. Inspect structured IncidentReport (replace <incident_id>)
  72. curl "http://localhost:8000/incidents/<incident_id>"
  73. # 3. Approve remediation (terminates blocking Postgres PID via MCP)
  74. curl -X POST "http://localhost:8000/incidents/<incident_id>/approve" \
  75. -H "Content-Type: application/json" \
  76. -d '{"approved":true,"operator":"oncall-engineer","comment":"Verified blocker PID"}'
  77. ```
  78. ## Automated Chaos Eval Suite
  79. Run the end-to-end evaluation harness to inject live faults, benchmark root-cause accuracy and exact PID precision, and write a JSON report to `evals/reports/`:
  80. ```bash
  81. uv run python evals/run_evals.py
  82. ```
  83. ## Standalone CLI & MCP Inspector
  84. * **Run the triage loop directly without FastAPI:**
  85. ```bash
  86. uv run python -m agent_core.loop
  87. ```
  88. * **Inspect MCP telemetry tools interactively:**
  89. ```bash
  90. uv run mcp dev src/mcp_servers/telemetry_server/server.py
  91. ```
  92. ## Project Layout
  93. ```
  94. sentinel-sre/
  95. ├── src/
  96. │ ├── agent_core/ # Raw Gemini control loop, E2B wrapper, Pydantic schemas
  97. │ ├── api/ # FastAPI webhook receiver & HITL approval endpoints
  98. │ ├── db/ # SQLite incident state & audit persistence
  99. │ └── mcp_servers/
  100. │ └── telemetry_server/ # Custom MCP 2.x server (DB health, log streaming, PID exec)
  101. ├── evals/ # Automated chaos evaluation harness & reports
  102. ├── testbed/ # Target FastAPI + Postgres stack & chaos scripts
  103. ├── pyproject.toml
  104. └── README.md
  105. ```

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.