Skip to content
Back to search

llamafarm

Deploy any AI model, agent, database, RAG, and pipeline locally or remotely in minutes

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/llama-farm-llamafarm/badge.svg)](/listings/llama-farm-llamafarm)

HTML

<a href="/listings/llama-farm-llamafarm"><img src="/api/listings/llama-farm-llamafarm/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

These steps run a script from the network that findsafeskills does not scan.

Option 1: Desktop App

Download the desktop app above and run it. No additional setup required.

Option 2: CLI + Development Mode

1. **Install the CLI**

macOS / Linux:

   curl -fsSL https://raw.githubusercontent.com/llama-farm/llamafarm/main/install.sh | bash

Windows (PowerShell):

   irm https://raw.githubusercontent.com/llama-farm/llamafarm/main/install.ps1 | iex

Or download directly from [releases](https://github.com/llama-farm/llamafarm/releases/latest).

2. **Create and run a project**

   lf init my-project      # Generates llamafarm.yaml
   lf start                # Starts services and opens Designer UI

3. **Chat with your AI**

   lf chat                           # Interactive chat
   lf chat "Hello, LlamaFarm!"       # One-off message

The Designer web interface is available at `http://localhost:14345`.

Option 3: Development from Source

git clone https://github.com/llama-farm/llamafarm.git
cd llamafarm

# Install Nx globally and initialize the workspace
npm install -g nx
nx init --useDotNxInstallation --interactive=false  # Required on first clone

# Start all services (run each in a separate terminal)
nx start server           # FastAPI server (port 14345)
nx start rag              # RAG worker for document processing
nx start universal-runtime # ML models, OCR, embeddings (port 11540)

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 482 lines · 2 findings marked

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # LlamaFarm - Edge AI for Everyone
  2. > Enterprise AI capabilities on your own hardware. No cloud required.
  3. [![License: Apache 2.0](https://img.shields.io/github/license/llama-farm/llamafarm)](LICENSE)
  4. [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
  5. [![Go 1.24+](https://img.shields.io/badge/go-1.24+-00ADD8.svg)](https://go.dev/dl/)
  6. [![Docs](https://img.shields.io/badge/docs-latest-4C51BF.svg)](https://docs.llamafarm.dev)
  7. [![Discord](https://img.shields.io/discord/1392890421771899026.svg)](https://discord.gg/RrAUXTCVNF)
  8. **LlamaFarm** is an open-source AI platform that runs entirely on your hardware. Build RAG applications, train custom classifiers, detect anomalies, and run document processing—all locally with complete privacy.
  9. - 🔒 **Complete Privacy** — Your data never leaves your device
  10. - 💰 **No API Costs** — Use open-source models without per-token fees
  11. - 🌐 **Offline Capable** — Works without internet once models are downloaded
  12. - ⚡ **Hardware Optimized** — Automatic GPU/NPU acceleration on Apple Silicon, NVIDIA, and AMD
  13. ### Desktop App Downloads
  14. Get started instantly — no command line required:
  15. | Platform | Download |
  16. |----------|----------|
  17. | **Mac (Universal)** | [Download](https://github.com/llama-farm/llamafarm/releases/latest/download/LlamaFarm-desktop-app-mac-universal.dmg) |
  18. | **Windows** | [Download](https://github.com/llama-farm/llamafarm/releases/latest/download/LlamaFarm-desktop-app-windows.exe) |
  19. | **Linux (x86_64)** | [Download](https://github.com/llama-farm/llamafarm/releases/latest/download/LlamaFarm-desktop-app-linux-x86_64.AppImage) |
  20. | **Linux (ARM64)** | [Download](https://github.com/llama-farm/llamafarm/releases/latest/download/LlamaFarm-desktop-app-linux-arm64.AppImage) |
  21. ---
  22. ### What Can You Build?
  23. | Capability | Description |
  24. |-----------|-------------|
  25. | **RAG (Retrieval-Augmented Generation)** | Ingest PDFs, docs, CSVs and query them with AI |
  26. | **Custom Classifiers** | Train text classifiers with 8-16 examples using SetFit |
  27. | **Anomaly Detection** | 12+ algorithms for batch and streaming anomaly detection |
  28. | **Tool Calling (MCP)** | Connect models to external tools via Model Context Protocol |
  29. | **OCR & Document Extraction** | Extract text and structured data from images and PDFs |
  30. | **Named Entity Recognition** | Find people, organizations, and locations |
  31. | **Multi-Model Runtime** | Switch between Ollama, OpenAI, vLLM, or local GGUF models |
  32. **Video demo (90 seconds):** https://youtu.be/W7MHGyN0MdQ
  33. ---
  34. ## Quickstart
  35. ### Option 1: Desktop App
  36. Download the desktop app above and run it. No additional setup required.
  37. ### Option 2: CLI + Development Mode
  38. 1. **Install the CLI**
  39. macOS / Linux:
  40. ```bash
  41. curl -fsSL https://raw.githubusercontent.com/llama-farm/llamafarm/main/install.sh | bash
    • Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
    • Line 59 on GitHub
  42. ```
  43. Windows (PowerShell):
  44. ```powershell
  45. irm https://raw.githubusercontent.com/llama-farm/llamafarm/main/install.ps1 | iex
  46. ```
  47. Or download directly from [releases](https://github.com/llama-farm/llamafarm/releases/latest).
  48. 2. **Create and run a project**
  49. ```bash
  50. lf init my-project # Generates llamafarm.yaml
  51. lf start # Starts services and opens Designer UI
  52. ```
  53. 3. **Chat with your AI**
  54. ```bash
  55. lf chat # Interactive chat
  56. lf chat "Hello, LlamaFarm!" # One-off message
  57. ```
  58. The Designer web interface is available at `http://localhost:14345`.
  59. ### Option 3: Development from Source
  60. ```bash
  61. git clone https://github.com/llama-farm/llamafarm.git
  62. cd llamafarm
  63. # Install Nx globally and initialize the workspace
  64. npm install -g nx
  65. nx init --useDotNxInstallation --interactive=false # Required on first clone
  66. # Start all services (run each in a separate terminal)
  67. nx start server # FastAPI server (port 14345)
  68. nx start rag # RAG worker for document processing
  69. nx start universal-runtime # ML models, OCR, embeddings (port 11540)
  70. ```
  71. ---
  72. ## Architecture
  73. LlamaFarm consists of three main services:
  74. | Service | Port | Purpose |
  75. |---------|------|---------|
  76. | **Server** | 14345 | FastAPI REST API, Designer web UI, project management |
  77. | **RAG Worker** | - | Celery worker for async document processing |
  78. | **Universal Runtime** | 11540 | ML model inference, embeddings, OCR, anomaly detection |
  79. All configuration lives in `llamafarm.yaml`—no scattered settings or hidden defaults.
  80. ---
  81. ## Runtime Options
  82. ### Universal Runtime (Recommended)
  83. The Universal Runtime provides access to HuggingFace models plus specialized ML capabilities:
  84. - **Text Generation** - Any HuggingFace text model
  85. - **Embeddings** - sentence-transformers and other embedding models
  86. - **OCR** - Text extraction from images/PDFs (Surya, EasyOCR, PaddleOCR, Tesseract)
  87. - **Document Extraction** - Forms, invoices, receipts via vision models
  88. - **Text Classification** - Pre-trained or custom models via SetFit
  89. - **Named Entity Recognition** - Extract people, organizations, locations
  90. - **Reranking** - Cross-encoder models for improved RAG quality
  91. - **Anomaly Detection** - Isolation Forest, One-Class SVM, Local Outlier Factor, Autoencoders
  92. ```yaml
  93. runtime:
  94. models:
  95. default:
  96. provider: universal
  97. model: Qwen/Qwen2.5-1.5B-Instruct
  98. base_url: http://127.0.0.1:11540/v1
  99. ```
  100. ### Ollama
  101. Simple setup for GGUF models with CPU/GPU acceleration:
  102. ```yaml
  103. runtime:
  104. models:
  105. default:
  106. provider: ollama
  107. model: qwen3:8b
  108. base_url: http://localhost:11434/v1
  109. ```
  110. ### OpenAI-Compatible
  111. Works with vLLM, Together, Mistral API, or any OpenAI-compatible endpoint:
  112. ```yaml
  113. runtime:
  114. models:
  115. default:
  116. provider: openai
  117. model: gpt-4o
  118. base_url: https://api.openai.com/v1
  119. api_key: ${OPENAI_API_KEY}
  120. ```
  121. ---
  122. ## Core Workflows
  123. ### CLI Commands
  124. | Task | Command |
  125. |------|---------|
  126. | Initialize project | `lf init my-project` |
  127. | Start services | `lf start` |
  128. | Interactive chat | `lf chat` |
  129. | One-off message | `lf chat "Your question"` |
  130. | List models | `lf models list` |
  131. | Use specific model | `lf chat --model powerful "Question"` |
  132. | Create dataset | `lf datasets create -s pdf_ingest -b main_db research` |
  133. | Upload files (auto-process by default) | `lf datasets upload research ./docs/*.pdf` |
  134. | Process dataset (if you skipped auto-process) | `lf datasets process research` |
  135. | Query RAG | `lf rag query --database main_db "Your query"` |
  136. | Check RAG health | `lf rag health` |
  137. ### RAG Pipeline
  138. 1. **Create a dataset** linked to a processing strategy and database
  139. 2. **Upload files** (PDF, DOCX, Markdown, TXT) — processing runs automatically unless you pass `--no-process`
  140. 3. **Process manually** only when you intentionally skipped auto-processing (e.g., large batches)
  141. 4. **Query** using semantic search with optional metadata filtering
  142. ```bash
  143. lf datasets create -s default -b main_db research
  144. lf datasets upload research ./papers/*.pdf # auto-processes by default
  145. # For large batches:
  146. # lf datasets upload research ./papers/*.pdf --no-process
  147. # lf datasets process research
  148. lf rag query --database main_db "What are the key findings?"
  149. ```
  150. ### Designer Web UI
  151. The Designer at `http://localhost:14345` provides:
  152. - **Project management** with briefs and quick actions
  153. - **Visual dataset management** with drag-and-drop uploads
  154. - **Database & RAG configuration** with built-in query testing
  155. - **Prompt engineering** with template variables and testing
  156. - **Interactive chat** with RAG toggle and retrieved context display
  157. - **Config editor** with syntax highlighting, validation, and auto-completion
  158. - Switch between visual Designer and raw YAML modes in any section
  159. See the [Designer Features Guide](docs/website/docs/designer/features.md) for details.
  160. ---
  161. ## Configuration
  162. `llamafarm.yaml` is the source of truth for each project:
  163. ```yaml
  164. version: v1
  165. name: my-assistant
  166. namespace: default
  167. # Multi-model configuration
  168. runtime:
  169. default_model: fast
  170. models:
  171. fast:
  172. description: "Fast local model"
  173. provider: universal
  174. model: Qwen/Qwen2.5-1.5B-Instruct
  175. base_url: http://127.0.0.1:11540/v1
  176. powerful:
  177. description: "More capable model"
  178. provider: universal
  179. model: Qwen/Qwen2.5-7B-Instruct
  180. base_url: http://127.0.0.1:11540/v1
  181. # System prompts
  182. prompts:
  183. - name: default
  184. messages:
  185. - role: system
  186. content: You are a helpful assistant.
  187. # RAG configuration
  188. rag:
  189. databases:
  190. - name: main_db
  191. type: ChromaStore
  192. default_embedding_strategy: default_embeddings
  193. default_retrieval_strategy: semantic_search
  194. embedding_strategies:
  195. - name: default_embeddings
  196. type: UniversalEmbedder
  197. config:
  198. model: sentence-transformers/all-MiniLM-L6-v2
  199. base_url: http://127.0.0.1:11540/v1
  200. retrieval_strategies:
  201. - name: semantic_search
  202. type: BasicSimilarityStrategy
  203. config:
  204. top_k: 5
  205. data_processing_strategies:
  206. - name: default
  207. parsers:
  208. - type: PDFParser_LlamaIndex
  209. config:
  210. chunk_size: 1000
  211. chunk_overlap: 100
  212. - type: MarkdownParser_Python
  213. config:
  214. chunk_size: 1000
  215. extractors: []
  216. # Dataset definitions
  217. datasets:
  218. - name: research
  219. data_processing_strategy: default
  220. database: main_db
  221. ```
  222. ### Environment Variable Substitution
  223. Use `${VAR}` syntax to inject secrets from `.env` files:
  224. ```yaml
  225. runtime:
  226. models:
  227. openai:
  228. api_key: ${OPENAI_API_KEY}
  229. # With default: ${OPENAI_API_KEY:-sk-default}
  230. # From specific file: ${file:.env.production:API_KEY}
  231. ```
  232. See the [Configuration Guide](docs/website/docs/configuration/index.md) for complete reference.
  233. ---
  234. ## REST API
  235. LlamaFarm provides an OpenAI-compatible REST API:
  236. **Chat Completions**
  237. ```bash
  238. curl -X POST http://localhost:14345/v1/projects/default/my-project/chat/completions \
  239. -H "Content-Type: application/json" \
  240. -d '{
  241. "messages": [{"role": "user", "content": "Hello"}],
  242. "stream": false,
  243. "rag_enabled": true
  244. }'
  245. ```
  246. **RAG Query**
  247. ```bash
  248. curl -X POST http://localhost:14345/v1/projects/default/my-project/rag/query \
  249. -H "Content-Type: application/json" \
  250. -d '{
  251. "query": "What are the requirements?",
  252. "database": "main_db",
  253. "top_k": 5
  254. }'
  255. ```
  256. See the [API Reference](docs/website/docs/api/index.md) for all endpoints.
  257. ---
  258. ## Specialized ML Capabilities
  259. The Universal Runtime provides endpoints beyond chat:
  260. ### OCR & Document Extraction
  261. ```bash
  262. curl -X POST http://localhost:14345/v1/vision/ocr \
  263. -F "file=@document.pdf" \
  264. -F "model=surya"
  265. ```
  266. ### Anomaly Detection
  267. LlamaFarm supports 12+ anomaly detection algorithms via PyOD, with both batch and streaming modes.
  268. ```bash
  269. # Train on normal data
  270. curl -X POST http://localhost:14345/v1/ml/anomaly/fit \
  271. -H "Content-Type: application/json" \
  272. -d '{"model": "sensor-detector", "backend": "ecod", "data": [[22.1], [23.5], ...]}'
  273. # Detect anomalies
  274. curl -X POST http://localhost:14345/v1/ml/anomaly/detect \
  275. -H "Content-Type: application/json" \
  276. -d '{"model": "sensor-detector", "data": [[22.0], [100.0], [23.0]], "threshold": 0.5}'
  277. # Streaming detection (handles cold start, auto-retraining, sliding windows)
  278. curl -X POST http://localhost:14345/v1/ml/anomaly/stream \
  279. -H "Content-Type: application/json" \
  280. -d '{"model": "live-sensor", "data": {"temperature": 72.5}, "backend": "ecod"}'
  281. ```
  282. **Available backends:** `ecod` (recommended), `isolation_forest`, `one_class_svm`, `local_outlier_factor`, `autoencoder`, `hbos`, `copod`, `knn`, `mcd`, `cblof`, `suod`, `loda`
  283. ### Text Classification & NER
  284. See the [Models Guide](docs/website/docs/models/index.md) for complete documentation.
  285. ### Tool Calling (MCP)
  286. Give models access to external tools via the Model Context Protocol:
  287. ```yaml
  288. # In llamafarm.yaml
  289. mcp:
  290. servers:
  291. - name: filesystem
  292. transport: stdio
  293. command: npx
  294. args: ['-y', '@modelcontextprotocol/server-filesystem', '/data']
  295. runtime:
  296. models:
  297. - name: assistant
  298. provider: ollama
  299. model: llama3.1:8b
  300. mcp_servers: [filesystem]
  301. ```
  302. LlamaFarm also exposes its own API as MCP tools for use with Claude Desktop, Cursor, and other MCP clients. See the [Tool Calling Guide](docs/website/docs/mcp/index.md).
  303. ---
  304. ## Examples
  305. | Example | Description | Location |
  306. |---------|-------------|----------|
  307. | **RAG Examples** | | |
  308. | Large Complex PDFs | Multi-megabyte planning ordinances | `examples/large_complex_rag/` |
  309. | Many Small Files | FDA correspondence letters | `examples/many_small_file_rag/` |
  310. | Mixed Formats | PDF, Markdown, HTML, text, and code | `examples/mixed_format_rag/` |
  311. | Quick Notes | Rapid smoke tests with small files | `examples/quick_rag/` |
  312. | **Anomaly Detection** | | |
  313. | Quick Start | Simplest anomaly detection example | `examples/anomaly/01_quick_start.py` |
  314. | Fraud Detection | Training, saving, loading models | `examples/anomaly/02_fraud_detection.py` |
  315. | Streaming Sensors | IoT monitoring with rolling features | `examples/anomaly/03_streaming_sensors.py` |
  316. | Backend Comparison | Compare all 12 algorithms | `examples/anomaly/04_backend_comparison.py` |
  317. | **Use Cases** | | |
  318. | FDA Letters Assistant | Regulatory document analysis | `examples/fda_rag/` |
  319. | Government Planning | Large ordinance documents | `examples/gov_rag/` |
  320. See [`examples/README.md`](examples/README.md) for setup instructions and the full list.
  321. ---
  322. ## Industry Use Cases
  323. LlamaFarm is used across industries for document analysis, monitoring, and fraud detection:
  324. - **[Pharmaceutical & Therapeutics](docs/website/docs/use-cases/pharmaceutical-fda.md)** — Analyze FDA correspondence, track regulatory questions
  325. - **[IoT Sensor Monitoring](docs/website/docs/use-cases/iot-sensor-monitoring.md)** — Real-time streaming anomaly detection with automatic retraining
  326. - **[Financial Fraud Detection](docs/website/docs/use-cases/financial-fraud-detection.md)** — Multi-stage fraud detection with velocity and behavioral patterns
  327. ---
  328. ## Development & Testing
  329. ```bash
  330. # Python server tests
  331. cd server && uv sync && uv run --group test python -m pytest
  332. # CLI tests
  333. cd cli && go test ./...
  334. # RAG tests
  335. cd rag && uv sync && uv run pytest tests/
  336. # Universal Runtime tests
  337. cd runtimes/universal && uv sync && uv run pytest tests/
  338. # Build docs
  339. nx build docs
  340. ```
  341. ---
  342. ## Extensibility
  343. - **Add runtimes** by implementing provider support and updating schema
  344. - **Add vector stores** by implementing store backends (Chroma, Qdrant, etc.)
  345. - **Add parsers** for new file formats (PDF, DOCX, HTML, CSV, etc.)
  346. - **Add extractors** for custom metadata extraction
  347. - **Add CLI commands** under `cli/cmd/`
  348. See the [Extending Guide](docs/website/docs/extending/index.md) for step-by-step instructions.
  349. ---
  350. ## Community & Support
  351. - [Discord](https://discord.gg/RrAUXTCVNF) - Chat with the team and community
  352. - [GitHub Issues](https://github.com/llama-farm/llamafarm/issues) - Bug reports and feature requests
  353. - [Discussions](https://github.com/llama-farm/llamafarm/discussions) - Ideas and proposals
  354. - [Contributing Guide](CONTRIBUTING.md) - Code style and contribution process
  355. ---
  356. ## License
  357. Licensed under the [Apache 2.0 License](LICENSE). See [CREDITS](CREDITS.md) for acknowledgments.
  358. ---
  359. Build locally. Deploy anywhere. Own your AI.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.