Skip to content
Back to search

hr-helper

Practice multi-agent HR assistant: LangGraph agents + our own MCP server over MySQL (Docker) + RAG on a mock handbook + evals + React UI

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/hari-codeio--hr-helper/badge.svg)](/listings/hari-codeio--hr-helper)

HTML

<a href="/listings/hari-codeio--hr-helper"><img src="/api/listings/hari-codeio--hr-helper/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 165 lines
  1. # Soft Suave HR Helper
  2. A small, production-style AI app. Employees ask HR questions ("Can I work from home this Friday?").
  3. The app answers from the **company handbook** (RAG) and the **employee's own records** (MySQL, reached through
  4. our own **MCP** server). A team of **LangGraph agents** does the work, and a **Checker** agent verifies every
  5. answer before the employee sees it.
  6. > Practice project. The handbook (`data/handbook.pdf`) and every employee are made up.
  7. ## What it uses, and why each part is needed
  8. | Part | Tool | Without it the app breaks because... |
  9. |---|---|---|
  10. | RAG | Chroma + local embeddings | The rules are in a private handbook the LLM has never seen (e.g. "Friday + Monday leave = 4 days") |
  11. | Our internal API | MCP server (Python) over MySQL | Grade, probation, notice period and balances are per employee and change daily |
  12. | Database | MySQL 8.4 in Docker | Holds the mock employees, balances and leave requests |
  13. | Agents | LangGraph + LangChain `create_agent` | The answer needs several steps; each depends on the last |
  14. | Multi-agent | HR Desk + 2 workers + Checker | Maker-checker: nothing reaches the employee unverified |
  15. | LLM | Gemini (free tier) + a backup model | The brain. The backup takes over when the main model is down |
  16. | API | FastAPI | One service the UI (or anything else) calls |
  17. | UI | React (Vite) | Mock login, my records, apply leave, chat with live agent steps |
  18. | Evals | 17 cases with exact answers | Proves it works, and shows the LLM without RAG gets it wrong |
  19. ## The flow of one question
  20. ```
  21. Browser (React) --POST /api/ask/stream, header X-Employee-Id: E002--> FastAPI (api/main.py)
  22. |
  23. LangGraph (agents/graph.py) v
  24. 1. HR Desk (orchestrator) reads the question, plans who to send -----+
  25. | |
  26. 2a. Policy Agent 2b. Records Agent <- run IN PARALLEL
  27. search_handbook get_my_profile / get_my_leave_balance / get_my_leave_requests
  28. | (rag/search.py) | (mcp_server/client.py -> MCP -> mcp_server/server.py -> MySQL)
  29. v v
  30. 3. Writer: answer + decision + cited clauses (structured JSON)
  31. 4. Checker: code checks (cited clause really retrieved?) + LLM check against the evidence
  32. |-- rejected and round < 3 --> back to 1 with the Checker's feedback
  33. v
  34. 5. Finish: verified answer, or "I could not verify, please ask HR" (never a guess)
  35. |
  36. Every step streams to the UI live; the full trace goes to traces/traces.jsonl
  37. ```
  38. Example: Ravi (on probation) asks "Can I work from home this Friday?"
  39. -> HR Desk sends both workers -> Records: `on_probation: true` -> Policy: clause 5.3
  40. -> Writer: **No**, "on probation you work from the office all 5 days (clause 5.3)" -> Checker approves.
  41. ## Run it (Windows, PowerShell, from this folder)
  42. ```powershell
  43. # 1. Database (Docker Desktop must be running). Wait until "healthy".
  44. docker compose up -d
  45. docker compose ps
  46. # 2. Python packages
  47. python -m venv .venv
  48. .venv\Scripts\Activate.ps1
  49. pip install -r requirements.txt
  50. copy .env.example .env # then put your GEMINI_API_KEY and passwords in .env
  51. # 3. Build the handbook index (once, and again when the PDF changes)
  52. python -m rag.ingest
  53. # 4. Start the API -> http://127.0.0.1:8000/docs
  54. uvicorn api.main:app --port 8000
  55. # 5. Start the UI (a second terminal) -> http://localhost:5173
  56. cd ui
  57. npm install
  58. npm run dev
  59. ```
  60. Or build the UI once (`cd ui; npm run build`) and FastAPI serves it at http://127.0.0.1:8000.
  61. Tests and evals:
  62. ```powershell
  63. pytest # 18 unit tests: no LLM, no Docker needed
  64. python -m evals.run_evals # 17 cases + no-RAG baseline (~20 min on the free tier)
  65. python -m evals.run_evals --only c01,c16 --no-baseline
  66. ```
  67. ## Folder map
  68. | Path | What it does |
  69. |---|---|
  70. | `config.py` | Every setting, read from `.env` in one place |
  71. | `docker-compose.yml` | MySQL in Docker on port **3307** (your own MySQL keeps 3306) |
  72. | `db/seed.sql` | Tables + 10 mock employees, each built to test one handbook rule |
  73. | `data/handbook.pdf` | The mock Soft Suave handbook (46 clauses) |
  74. | `mcp_server/server.py` | **Our internal API**: MCP tools over MySQL (parameterized SQL only) |
  75. | `mcp_server/rules.py` | Hard leave rules in plain code (an LLM can be tricked; this can't) |
  76. | `mcp_server/client.py` | Starts the MCP server and calls its tools |
  77. | `rag/ingest.py` | PDF -> one chunk per clause (tables kept as rows) -> Chroma |
  78. | `rag/search.py` | Find the closest clauses to a question |
  79. | `agents/llm.py` | Gemini: main + backup model, retries, one shared rate limiter |
  80. | `agents/prompts.py` | Every prompt, versioned |
  81. | `agents/workers.py` | Policy Agent and Records Agent (ReAct, step limit) |
  82. | `agents/graph.py` | The LangGraph flow: state, nodes, parallel edges, the Checker loop |
  83. | `agents/checks.py` | Code checks run before the LLM Checker |
  84. | `agents/runner.py` | Runs one question, streams steps, writes the trace |
  85. | `api/main.py` | FastAPI endpoints and the mock login |
  86. | `ui/src/` | React: `App.jsx`, `api.js`, `components/` |
  87. | `evals/` | `cases.jsonl` (17 cases) and `run_evals.py` |
  88. | `tests/` | Unit tests for the code-only parts |
  89. ## Production habits in this small app (good interview talking points)
  90. 1. **Identity from login, not from the prompt.** The Records Agent's tools have no `emp_id` parameter; the id is
  91. fixed from the login. Ravi asking "What is Meena's balance?" can't work, whatever the LLM does.
  92. 2. **The LLM explains, code decides.** `apply_leave` is a write tool. Only a human clicking the form calls it, and
  93. `mcp_server/rules.py` decides. The agents are never given it.
  94. 3. **Maker-checker with a loop limit.** A Checker verifies every answer; at most 3 rounds, then "ask HR", never a guess.
  95. 4. **Code checks before LLM checks.** "Cited clause 3.4 but never retrieved it" is caught for free, exactly.
  96. 5. **Structured outputs.** Every LLM decision is JSON in a fixed shape (Pydantic), so the evals can compare exactly.
  97. 6. **Budgets everywhere.** Worker step limit, graph recursion limit, 30 s call timeout, shared rate limiter.
  98. 7. **Resilience.** Node retries with backoff, plus a backup model. (While building this, `gemini-3.1-flash-lite`
  99. returned `503 high demand`; the backup switch is the fix.)
  100. 8. **Safe errors.** The MCP server sends only deliberate `ToolError` messages; crash details stay in its log.
  101. 9. **Least privilege in the database.** The app uses `hr_app`, not root, and only parameterized SQL.
  102. 10. **Tracing.** Every question: steps, tool calls, tokens, time, model and prompt version -> `traces/traces.jsonl`
  103. (and Langfuse, if `LANGFUSE_ENABLED=1`).
  104. 11. **Evals with exact answers**, plus safety cases (prompt injection, data leak) and a no-RAG baseline.
  105. ## Honest limits
  106. - **Slow on the free tier:** about 7 LLM calls per question at about 12 calls a minute, so 30-60 s per answer.
  107. A paid tier or a faster model fixes this. The UI streams the steps so the wait is visible.
  108. - **Mock login:** the `X-Employee-Id` header stands in for a real SSO/JWT login.
  109. - **Simplified rules in code:** `rules.py` skips festival holidays and the sandwich rule when counting days.
  110. - **Small eval set:** 17 cases show the method. A real release would use hundreds, taken from real questions.
  111. ## Eval results
  112. Latest run (2026-11-21, `gemini-3.5-flash-lite` with backup `gemini-3.6-flash`, prompts v2). Full table and
  113. every answer: [`evals/results.md`](evals/results.md).
  114. | Metric | HR Helper | LLM with NO handbook |
  115. |---|---|---|
  116. | Decision + facts correct | **17/17** | 5/17 |
  117. | Decision label correct | 17/17 | 8/17 |
  118. | Right clause cited | 17/17 | - |
  119. | Right agents used (trajectory) | 17/17 | - |
  120. | Safety: prompt injection + data leak | 2/2 (nothing written to the DB) | - |
  121. | Verified by the Checker | 17/17 | - |
  122. | Raw retrieval hit@3 (question as typed) | 10/12 | - |
  123. | Average per question | 1.1 rounds, ~4,800 tokens, ~52 s | 1 call |
  124. **How we got there (each run found something real):**
  125. | Run | Result | What it found | Fix |
  126. |---|---|---|---|
  127. | 1 | crashed at case 12 (11/11 passed before) | Google timed out; LangGraph's default retry rule skips `TimeoutError` | own retry rule (`is_temporary_error`), longer backup timeout, evals survive one failed case |
  128. | 2 | 15/17 | (a) two parallel searches opened Chroma at once -> `KeyError`; (b) "Do I get a comp-off?" answered correctly but labelled `info` | lock + warm-up in `rag/search.py`; Writer rule for yes/no questions (prompts v2) |
  129. | 3 | 16/17, then 17/17 | the one "failure" was **our check**: it demanded the word "Independence", the answer said "national holiday" | loosened the check, re-graded the saved answers with `--rescore` (no new LLM calls) |
  130. **Read these numbers carefully:**
  131. - 17 cases is a small set, and prompts v2 was written after seeing run 2's failures. The fix is a general
  132. rule, not case-specific, but new cases (written before looking at answers) are the real test.
  133. - Raw retrieval finds the right clause in the top 3 for 10 of 12 questions. The other two (c03, c04) still
  134. passed because the Policy Agent rewrote its own search queries: the reason it is an agent, not one search.
  135. - Two cases (c12, c13) needed a 2nd round: the Checker rejected a draft because the employee's records had not
  136. been fetched (the internet allowance depends on grade and confirmation). That is the maker-checker loop working.
  137. - Time per question varies a lot on the free tier (25 s to 170 s) depending on Google's load.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.