Back to search
hr-helper
MCP serverGitHub
Practice multi-agent HR assistant: LangGraph agents + our own MCP server over MySQL (Docker) + RAG on a mock handbook + evals + React UI
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Badge for your README
Markdown
[](/listings/hari-codeio--hr-helper)HTML
<a href="/listings/hari-codeio--hr-helper"><img src="/api/listings/hari-codeio--hr-helper/badge.svg" alt="findsafeskills safety rating" /></a>Install
i
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
README.md · README · 165 lines
- # Soft Suave HR Helper
- A small, production-style AI app. Employees ask HR questions ("Can I work from home this Friday?").
- The app answers from the **company handbook** (RAG) and the **employee's own records** (MySQL, reached through
- our own **MCP** server). A team of **LangGraph agents** does the work, and a **Checker** agent verifies every
- answer before the employee sees it.
- > Practice project. The handbook (`data/handbook.pdf`) and every employee are made up.
- ## What it uses, and why each part is needed
- | Part | Tool | Without it the app breaks because... |
- |---|---|---|
- | RAG | Chroma + local embeddings | The rules are in a private handbook the LLM has never seen (e.g. "Friday + Monday leave = 4 days") |
- | Our internal API | MCP server (Python) over MySQL | Grade, probation, notice period and balances are per employee and change daily |
- | Database | MySQL 8.4 in Docker | Holds the mock employees, balances and leave requests |
- | Agents | LangGraph + LangChain `create_agent` | The answer needs several steps; each depends on the last |
- | Multi-agent | HR Desk + 2 workers + Checker | Maker-checker: nothing reaches the employee unverified |
- | LLM | Gemini (free tier) + a backup model | The brain. The backup takes over when the main model is down |
- | API | FastAPI | One service the UI (or anything else) calls |
- | UI | React (Vite) | Mock login, my records, apply leave, chat with live agent steps |
- | Evals | 17 cases with exact answers | Proves it works, and shows the LLM without RAG gets it wrong |
- ## The flow of one question
- ```
- Browser (React) --POST /api/ask/stream, header X-Employee-Id: E002--> FastAPI (api/main.py)
- |
- LangGraph (agents/graph.py) v
- 1. HR Desk (orchestrator) reads the question, plans who to send -----+
- | |
- 2a. Policy Agent 2b. Records Agent <- run IN PARALLEL
- search_handbook get_my_profile / get_my_leave_balance / get_my_leave_requests
- | (rag/search.py) | (mcp_server/client.py -> MCP -> mcp_server/server.py -> MySQL)
- v v
- 3. Writer: answer + decision + cited clauses (structured JSON)
- 4. Checker: code checks (cited clause really retrieved?) + LLM check against the evidence
- |-- rejected and round < 3 --> back to 1 with the Checker's feedback
- v
- 5. Finish: verified answer, or "I could not verify, please ask HR" (never a guess)
- |
- Every step streams to the UI live; the full trace goes to traces/traces.jsonl
- ```
- Example: Ravi (on probation) asks "Can I work from home this Friday?"
- -> HR Desk sends both workers -> Records: `on_probation: true` -> Policy: clause 5.3
- -> Writer: **No**, "on probation you work from the office all 5 days (clause 5.3)" -> Checker approves.
- ## Run it (Windows, PowerShell, from this folder)
- ```powershell
- # 1. Database (Docker Desktop must be running). Wait until "healthy".
- docker compose up -d
- docker compose ps
- # 2. Python packages
- python -m venv .venv
- .venv\Scripts\Activate.ps1
- pip install -r requirements.txt
- copy .env.example .env # then put your GEMINI_API_KEY and passwords in .env
- # 3. Build the handbook index (once, and again when the PDF changes)
- python -m rag.ingest
- # 4. Start the API -> http://127.0.0.1:8000/docs
- uvicorn api.main:app --port 8000
- # 5. Start the UI (a second terminal) -> http://localhost:5173
- cd ui
- npm install
- npm run dev
- ```
- Or build the UI once (`cd ui; npm run build`) and FastAPI serves it at http://127.0.0.1:8000.
- Tests and evals:
- ```powershell
- pytest # 18 unit tests: no LLM, no Docker needed
- python -m evals.run_evals # 17 cases + no-RAG baseline (~20 min on the free tier)
- python -m evals.run_evals --only c01,c16 --no-baseline
- ```
- ## Folder map
- | Path | What it does |
- |---|---|
- | `config.py` | Every setting, read from `.env` in one place |
- | `docker-compose.yml` | MySQL in Docker on port **3307** (your own MySQL keeps 3306) |
- | `db/seed.sql` | Tables + 10 mock employees, each built to test one handbook rule |
- | `data/handbook.pdf` | The mock Soft Suave handbook (46 clauses) |
- | `mcp_server/server.py` | **Our internal API**: MCP tools over MySQL (parameterized SQL only) |
- | `mcp_server/rules.py` | Hard leave rules in plain code (an LLM can be tricked; this can't) |
- | `mcp_server/client.py` | Starts the MCP server and calls its tools |
- | `rag/ingest.py` | PDF -> one chunk per clause (tables kept as rows) -> Chroma |
- | `rag/search.py` | Find the closest clauses to a question |
- | `agents/llm.py` | Gemini: main + backup model, retries, one shared rate limiter |
- | `agents/prompts.py` | Every prompt, versioned |
- | `agents/workers.py` | Policy Agent and Records Agent (ReAct, step limit) |
- | `agents/graph.py` | The LangGraph flow: state, nodes, parallel edges, the Checker loop |
- | `agents/checks.py` | Code checks run before the LLM Checker |
- | `agents/runner.py` | Runs one question, streams steps, writes the trace |
- | `api/main.py` | FastAPI endpoints and the mock login |
- | `ui/src/` | React: `App.jsx`, `api.js`, `components/` |
- | `evals/` | `cases.jsonl` (17 cases) and `run_evals.py` |
- | `tests/` | Unit tests for the code-only parts |
- ## Production habits in this small app (good interview talking points)
- 1. **Identity from login, not from the prompt.** The Records Agent's tools have no `emp_id` parameter; the id is
- fixed from the login. Ravi asking "What is Meena's balance?" can't work, whatever the LLM does.
- 2. **The LLM explains, code decides.** `apply_leave` is a write tool. Only a human clicking the form calls it, and
- `mcp_server/rules.py` decides. The agents are never given it.
- 3. **Maker-checker with a loop limit.** A Checker verifies every answer; at most 3 rounds, then "ask HR", never a guess.
- 4. **Code checks before LLM checks.** "Cited clause 3.4 but never retrieved it" is caught for free, exactly.
- 5. **Structured outputs.** Every LLM decision is JSON in a fixed shape (Pydantic), so the evals can compare exactly.
- 6. **Budgets everywhere.** Worker step limit, graph recursion limit, 30 s call timeout, shared rate limiter.
- 7. **Resilience.** Node retries with backoff, plus a backup model. (While building this, `gemini-3.1-flash-lite`
- returned `503 high demand`; the backup switch is the fix.)
- 8. **Safe errors.** The MCP server sends only deliberate `ToolError` messages; crash details stay in its log.
- 9. **Least privilege in the database.** The app uses `hr_app`, not root, and only parameterized SQL.
- 10. **Tracing.** Every question: steps, tool calls, tokens, time, model and prompt version -> `traces/traces.jsonl`
- (and Langfuse, if `LANGFUSE_ENABLED=1`).
- 11. **Evals with exact answers**, plus safety cases (prompt injection, data leak) and a no-RAG baseline.
- ## Honest limits
- - **Slow on the free tier:** about 7 LLM calls per question at about 12 calls a minute, so 30-60 s per answer.
- A paid tier or a faster model fixes this. The UI streams the steps so the wait is visible.
- - **Mock login:** the `X-Employee-Id` header stands in for a real SSO/JWT login.
- - **Simplified rules in code:** `rules.py` skips festival holidays and the sandwich rule when counting days.
- - **Small eval set:** 17 cases show the method. A real release would use hundreds, taken from real questions.
- ## Eval results
- Latest run (2026-11-21, `gemini-3.5-flash-lite` with backup `gemini-3.6-flash`, prompts v2). Full table and
- every answer: [`evals/results.md`](evals/results.md).
- | Metric | HR Helper | LLM with NO handbook |
- |---|---|---|
- | Decision + facts correct | **17/17** | 5/17 |
- | Decision label correct | 17/17 | 8/17 |
- | Right clause cited | 17/17 | - |
- | Right agents used (trajectory) | 17/17 | - |
- | Safety: prompt injection + data leak | 2/2 (nothing written to the DB) | - |
- | Verified by the Checker | 17/17 | - |
- | Raw retrieval hit@3 (question as typed) | 10/12 | - |
- | Average per question | 1.1 rounds, ~4,800 tokens, ~52 s | 1 call |
- **How we got there (each run found something real):**
- | Run | Result | What it found | Fix |
- |---|---|---|---|
- | 1 | crashed at case 12 (11/11 passed before) | Google timed out; LangGraph's default retry rule skips `TimeoutError` | own retry rule (`is_temporary_error`), longer backup timeout, evals survive one failed case |
- | 2 | 15/17 | (a) two parallel searches opened Chroma at once -> `KeyError`; (b) "Do I get a comp-off?" answered correctly but labelled `info` | lock + warm-up in `rag/search.py`; Writer rule for yes/no questions (prompts v2) |
- | 3 | 16/17, then 17/17 | the one "failure" was **our check**: it demanded the word "Independence", the answer said "national holiday" | loosened the check, re-graded the saved answers with `--rescore` (no new LLM calls) |
- **Read these numbers carefully:**
- - 17 cases is a small set, and prompts v2 was written after seeing run 2's failures. The fix is a general
- rule, not case-specific, but new cases (written before looking at answers) are the real test.
- - Raw retrieval finds the right clause in the top 3 for 10 of 12 questions. The other two (c03, c04) still
- passed because the Policy Agent rewrote its own search queries: the reason it is an agent, not one search.
- - Two cases (c12, c13) needed a 2nd round: the Checker rejected a draft because the employee's records had not
- been fetched (the internet allowance depends on grade and confirmation). That is the maker-checker loop working.
- - Time per question varies a lot on the free tier (25 s to 170 s) depending on Google's load.
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.