Skip to content
Back to search

dbt-llm-agent

LLM based AI Agent to automate Data Analysis for dbt projects with remote MCP server

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/pragunbhutani--dbt-llm-agent/badge.svg)](/listings/pragunbhutani--dbt-llm-agent)

HTML

<a href="/listings/pragunbhutani--dbt-llm-agent"><img src="/api/listings/pragunbhutani--dbt-llm-agent/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

# ① clone & prepare env file
$ git clone https://github.com/pragunbhutani/ragstar.git && cd ragstar
$ cp .env.example .env && ${EDITOR:-vi} .env  # ⇒ edit just the vars shown below

# ② build & run everything
$ docker compose up --build -d

When the containers are healthy:

- Frontend: http://localhost:3000 (Next.js) - API: http://localhost:8000 (Django/DRF) - Flower: http://localhost:5555 (background tasks)

Run first-time Django tasks:

# inside the running backend container
$ docker compose exec backend-django \
    uv run python manage.py migrate

🎉 That's it — open http://localhost:3000, sign up for a new account and you're ready to start using Ragstar.

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 391 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # Ragstar — AI Data Analyst for dbt Projects
  2. [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
  3. [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
  4. [![TypeScript](https://img.shields.io/badge/TypeScript-5.x-blue.svg)](https://www.typescriptlang.org/)
  5. [![Docker Compose](https://img.shields.io/badge/built%20with-Docker%20Compose-blue.svg)](https://docs.docker.com/compose/)
  6. > **Ragstar is in public βeta.** Expect rapid changes and occasional rough edges.
  7. ---
  8. ## 1. What is Ragstar?
  9. Ragstar connects to your **dbt** project, builds a knowledge base from models & documentation, and lets everyone ask data-related questions in plain English via a beautiful web dashboard or Slack. Under the hood Ragstar combines:
  10. - PostgreSQL + `pgvector` for fast similarity search
  11. - Embeddings + LLMs (OpenAI, Anthropic, etc.) for reasoning
  12. - A modern **Next.js** frontend & **Django** backend
  13. ### Screenshots
  14. #### Dashboard
  15. ![Dashboard](docs/dashboard.png)
  16. #### Integrations
  17. ![Integrations](docs/integrations.png)
  18. #### Settings
  19. ![Settings](docs/settings.png)
  20. ---
  21. ## 2. Quick start (🚀 2 commands)
  22. ```bash
  23. # ① clone & prepare env file
  24. $ git clone https://github.com/pragunbhutani/ragstar.git && cd ragstar
  25. $ cp .env.example .env && ${EDITOR:-vi} .env # ⇒ edit just the vars shown below
  26. # ② build & run everything
  27. $ docker compose up --build -d
  28. ```
  29. When the containers are healthy:
  30. - Frontend: http://localhost:3000 (Next.js)
  31. - API: http://localhost:8000 (Django/DRF)
  32. - Flower: http://localhost:5555 (background tasks)
  33. Run first-time Django tasks:
  34. ```bash
  35. # inside the running backend container
  36. $ docker compose exec backend-django \
  37. uv run python manage.py migrate
  38. ```
  39. 🎉 That's it — open http://localhost:3000, sign up for a new account and you're ready to start using Ragstar.
  40. ---
  41. ## 3. Environment variables
  42. Only a handful of variables are truly **required** for a local/dev install. The rest are advanced overrides.
  43. Ragstar keeps the default stack as lightweight as possible.
  44. For a **local `docker compose` run you only need 3 variables** – everything else has sane fall-backs.
  45. ### 3.1 Required for local/dev
  46. | Var | Example | Purpose |
  47. | --------------------- | ------------------------- | ---------------------------------------------------- |
  48. | `NEXTAUTH_SECRET` | `openssl rand -base64 32` | Secret used by **next-auth** to sign session cookies |
  49. | `NEXTAUTH_URL` | `http://localhost:3000` | Public URL where the frontend is reachable |
  50. | `NEXT_PUBLIC_API_URL` | `http://localhost:8000` | Public URL of the Django API exposed to the browser |
  51. Create a `.env` file in the repo root and paste the three lines above (adjust URLs if you changed the ports).
  52. ### 3.2 Common overrides (optional)
  53. | Var | Default | When you might set it |
  54. | ------------------------------------------------------------------ | ---------------------------- | ------------------------------------------------------------------------------------------------------ |
  55. | `INTERNAL_API_URL` | `http://backend-django:8000` | Only needed when the frontend talks to the backend across Docker networks or remote hosts. |
  56. | `ENVIRONMENT` | `local` | Switch between `local`, `development`, `production` behaviour inside Django settings. |
  57. | `APP_HOST` | — | Extra hostname to append to Django `ALLOWED_HOSTS` & CORS lists, e.g. your public Ngrok / Vercel host. |
  58. | `DATABASE_URL` | auto-generated by compose | Point the backend & worker to your own Postgres instance. |
  59. | `CELERY_BROKER_URL` | `redis://redis:6379/0` | Use an external Redis / RabbitMQ instead of the bundled one. |
  60. | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION` | not set | **Only required when you disable LocalStack and want to store secrets in real AWS Parameter Store.** |
  61. ### 3.3 LLM Provider API Keys
  62. You need to provide API keys for all the LLM providers you plan on using.
  63. ```bash
  64. LLM_OPENAI_API_KEY=...
  65. LLM_ANTHROPIC_API_KEY=...
  66. LLM_GOOGLE_API_KEY=...
  67. ```
  68. ### 3.4 Postgres defaults (used by compose)
  69. ```bash
  70. POSTGRES_DB=ragstar
  71. POSTGRES_USER=user
  72. POSTGRES_PASSWORD=password
  73. POSTGRES_PORT=5432
  74. ```
  75. Set a single `EXTERNAL_POSTGRES_URL=<url>` to BYO Postgres (with the `pgvector` extension).
  76. ---
  77. ## 4. First-run onboarding
  78. After logging into the dashboard you'll be guided through these steps:
  79. 1. **Add a dbt project** → _Projects › New_ (dbt **Cloud** recommended — just paste the service token). GitHub or local zip upload also supported.
  80. 2. **Pick the dbt models you want to use for answering questions** → choose which model to use for ∙ questions ∙ embeddings ∙ SQL verification.
  81. 3. **Configure Slack** (optional)
  82. - Go to _Integrations › Slack_.
  83. - Follow the inline manifest to create a Slack app.
  84. - Paste **Bot Token**, **Signing Secret**, **App Token**.
  85. 4. **Ask questions!** Use the chat on the dashboard or `/ask` in Slack.
  86. > Other integrations (Metabase, Snowflake, MCP) are available under _Integrations_ but currently **βeta / experimental**. MCP server is temporarily disabled while we stabilise streaming support.
  87. ---
  88. ## 5 Managing the stack
  89. Common operations are wrapped in one-liners:
  90. ```bash
  91. # shell into backend or frontend
  92. $ docker compose exec backend-django bash
  93. $ docker compose exec frontend-nextjs sh
  94. # tail logs
  95. $ docker compose logs -f backend-django
  96. # stop / remove containers
  97. docker compose down # keep volumes
  98. docker compose down -v # destroy DB
  99. ```
  100. ---
  101. ## 6. Local dev without Docker (advanced)
  102. 1. Install **Python 3.10+**, **Node 18+**, **uv**, **pnpm**, and Postgres16+ with `pgvector`.
  103. 2. `uv venv && source .venv/bin/activate && uv pip install -e backend_django/`
  104. 3. `pnpm install --filter frontend_nextjs`
  105. 4. Start services in two terminals:
  106. - **Backend** — `cd backend_django && uv run python manage.py runserver 0.0.0.0:8000`
  107. - **Frontend** — `cd frontend_nextjs && pnpm dev`
  108. 5. Export the same env vars listed above.
  109. Docker is strongly recommended unless you're hacking on the codebase itself.
  110. ---
  111. ## 7. Contributing
  112. We 💛 community PRs. Please file an issue first for major changes. Make sure `ruff`, `black`, `mypy`, and `eslint` pass before opening a pull request.
  113. ---
  114. ---
  115. ## 8. MCP Server Integration (Self-Hosted Only)
  116. Ragstar includes a **Model Context Protocol (MCP)** server that allows LLM clients like Claude.ai to directly access your dbt knowledge base. The MCP server provides secure, OAuth-authenticated access to your dbt models and project information.
  117. > **⚠️ Important:** The MCP server can only be used with **self-hosted/open source** deployments of Ragstar. This is due to the **1:1 relationship** between an MCP client and server — each client needs its own dedicated server instance.
  118. ### 8.1 What is the MCP Server?
  119. The MCP server acts as a bridge between LLM clients (like Claude.ai) and your Ragstar knowledge base. It provides:
  120. - **Secure OAuth 2.0 authentication** with PKCE
  121. - **Organization-scoped access** to dbt models and projects
  122. - **Real-time data access** without manual imports
  123. - **Semantic search** capabilities across your dbt documentation
  124. - **Detailed model information** including SQL, lineage, and metadata
  125. ### 8.2 Available MCP Tools
  126. The MCP server exposes these tools to LLM clients:
  127. - `list_dbt_models` — Browse and filter dbt models by project, schema, or materialization
  128. - `search_dbt_models` — Semantic search for relevant models using natural language
  129. - `get_model_details` — Get detailed information about specific models including SQL and lineage
  130. - `get_project_summary` — Overview of connected dbt projects and their structure
  131. ### 8.3 Setting Up the MCP Server
  132. #### Prerequisites
  133. 1. **Self-hosted Ragstar** running with Docker Compose
  134. 2. **dbt project connected** and models loaded in your knowledge base
  135. 3. **LLM client** that supports MCP (e.g., Claude.ai, ChatGPT with MCP support)
  136. #### Configuration
  137. The MCP server runs on port `8080` by default. Add these environment variables to your `.env` file:
  138. ```bash
  139. # MCP Server Configuration
  140. MCP_AUTHORIZATION_BASE_URL=http://localhost:8000 # Your Django backend URL
  141. DJANGO_BACKEND_URL=http://localhost:8000 # Backend URL for MCP server
  142. ALLOWED_ORIGINS=* # Or specific origins for security
  143. ```
  144. #### Starting the MCP Server
  145. The MCP server is included in the Docker Compose stack:
  146. ```bash
  147. # Start all services including MCP server
  148. docker compose up -d
  149. # Check MCP server health
  150. curl http://localhost:8080/health
  151. ```
  152. ### 8.4 OAuth 2.0 Flow Explanation
  153. The MCP server implements a complete OAuth 2.0 authorization flow with organization-scoped access. Here's how it works:
  154. #### Flow Overview
  155. ```mermaid
  156. sequenceDiagram
  157. participant Claude as Claude.ai
  158. participant MCP as MCP Server
  159. participant Django as Django Backend
  160. participant User as User Browser
  161. Claude->>MCP: 1. Request OAuth metadata
  162. MCP->>Django: 2. Proxy metadata request
  163. Django->>MCP: 3. Return OAuth configuration
  164. MCP->>Claude: 4. OAuth server metadata
  165. Claude->>MCP: 5. Authorization request (with PKCE)
  166. MCP->>Django: 6. Proxy authorization request
  167. Django->>User: 7. Redirect to frontend login
  168. User->>Django: 8. Authenticate via Next.js
  169. Django->>MCP: 9. Authorization code
  170. MCP->>Claude: 10. Return authorization code
  171. Claude->>MCP: 11. Exchange code for tokens
  172. MCP->>Django: 12. Validate and exchange
  173. Django->>MCP: 13. Access & refresh tokens
  174. MCP->>Claude: 14. JWT tokens
  175. Claude->>MCP: 15. API calls with Bearer token
  176. MCP->>Django: 16. Validate token & get user context
  177. Django->>MCP: 17. Organization-scoped data
  178. MCP->>Claude: 18. Formatted response
  179. ```
  180. #### Key Components
  181. 1. **OAuth Metadata Discovery**: MCP server exposes RFC 8414 compliant metadata endpoints
  182. 2. **PKCE Security**: Uses Proof Key for Code Exchange for enhanced security
  183. 3. **Auto-Registration**: Automatically registers new OAuth clients (like Claude.ai)
  184. 4. **Organization Scoping**: All data access is automatically scoped to the user's organization
  185. 5. **JWT Tokens**: Secure, stateless authentication using JSON Web Tokens
  186. #### Security Features
  187. - **PKCE (Proof Key for Code Exchange)**: Prevents authorization code interception attacks
  188. - **Organization Isolation**: Users can only access their organization's data
  189. - **Token Validation**: All API calls require valid JWT tokens
  190. - **Automatic Expiry**: Access tokens expire after 1 hour, refresh tokens after 7 days
  191. ### 8.5 Connecting Claude.ai to Your MCP Server
  192. #### Step 1: Configure MCP in Claude.ai
  193. 1. Go to Claude.ai settings
  194. 2. Find the "MCP Servers" or "Model Context Protocol" section
  195. 3. Add a new server with these details:
  196. - **Server URL**: `http://localhost:8080` (or your public URL)
  197. - **OAuth**: Enable OAuth 2.0 authentication
  198. - **Auto-discovery**: Enable to automatically discover capabilities
  199. #### Step 2: Authorize the Connection
  200. 1. Claude.ai will redirect you to your Ragstar login page
  201. 2. Sign in with your Ragstar account
  202. 3. You'll be redirected back to Claude.ai with authorization
  203. 4. The connection will be linked to your organization
  204. #### Step 3: Start Using MCP Tools
  205. You can now ask Claude.ai to:
  206. - "List all my dbt models"
  207. - "Search for revenue-related models"
  208. - "Show me details about the customer_metrics model"
  209. - "What dbt projects do I have connected?"
  210. ### 8.6 Example MCP Interactions
  211. #### Listing Models
  212. ```
  213. You: "What dbt models do I have available?"
  214. Claude: [Calls list_dbt_models tool]
  215. Claude: "You have 23 dbt models across 2 projects:
  216. - analytics_prod: 15 models (staging, marts schemas)
  217. - marketing_analytics: 8 models (staging, reporting schemas)
  218. ..."
  219. ```
  220. #### Searching Models
  221. ```
  222. You: "Find models related to customer revenue"
  223. Claude: [Calls search_dbt_models with "customer revenue"]
  224. Claude: "I found 3 relevant models:
  225. - customer_revenue_monthly (similarity: 0.95)
  226. - customer_ltv_calculation (similarity: 0.87)
  227. - revenue_attribution (similarity: 0.82)
  228. ..."
  229. ```
  230. #### Getting Model Details
  231. ```
  232. You: "Show me the SQL for customer_revenue_monthly"
  233. Claude: [Calls get_model_details for "customer_revenue_monthly"]
  234. Claude: "Here's the customer_revenue_monthly model:
  235. - Schema: marts
  236. - Materialization: table
  237. - SQL: SELECT customer_id, DATE_TRUNC('month', order_date) as month, SUM(amount) as revenue FROM..."
  238. ```
  239. ### 8.7 Troubleshooting MCP Connection
  240. #### Common Issues
  241. 1. **Authentication Fails**: Check that your Ragstar backend is accessible and you're logged in
  242. 2. **No Models Found**: Ensure your dbt project is connected and models are loaded
  243. 3. **Permission Errors**: Verify the user has access to the organization's dbt projects
  244. 4. **Network Issues**: Check firewall settings and port accessibility
  245. #### Debug Commands
  246. ```bash
  247. # Check MCP server health
  248. curl http://localhost:8080/health
  249. # Test OAuth metadata
  250. curl http://localhost:8080/.well-known/oauth-authorization-server
  251. # View MCP server logs
  252. docker compose logs -f mcp-server
  253. ```
  254. ### 8.8 Production Deployment
  255. For production use:
  256. 1. **Use HTTPS**: Configure SSL certificates for secure connections
  257. 2. **Restrict Origins**: Set specific allowed origins instead of `*`
  258. 3. **Monitor Usage**: Track OAuth token usage and API calls
  259. 4. **Scale Considerations**: Each client needs its own server instance
  260. ### 8.9 Limitations
  261. - **1:1 Client-Server Relationship**: Each MCP client needs its own server instance
  262. - **Organization Scoping**: Users can only access their own organization's data
  263. - **Token Expiry**: Access tokens expire after 1 hour (refresh tokens after 7 days)
  264. - **Self-Hosted Only**: Not available in hosted/SaaS deployments
  265. ---
  266. ## 9. Contributing
  267. We 💛 community PRs. Please file an issue first for major changes. Make sure `ruff`, `black`, `mypy`, and `eslint` pass before opening a pull request.
  268. ---
  269. ## 10. License
  270. Ragstar is released under the MIT License — see [LICENSE](./LICENSE).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.