Skip to content
Back to search

End-to-End-Agentic-Ai-Automation-Lab

This repository contains hands-on projects, code examples, and deployment workflows. Explore multi-agent systems, LangChain, LangGraph, AutoGen, CrewAI, RAG, MCP, automation with n8n, and scalable agent deployment using Docker, AWS, and BentoML.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/MDalamin5/End-to-End-Agentic-Ai-Automation-Lab with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/MDalamin5/End-to-End-Agentic-Ai-Automation-Lab'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/mdalamin5--end-to-end-agentic-ai-automation-lab/badge.svg)](/listings/mdalamin5--end-to-end-agentic-ai-automation-lab)

HTML

<a href="/listings/mdalamin5--end-to-end-agentic-ai-automation-lab"><img src="/api/listings/mdalamin5--end-to-end-agentic-ai-automation-lab/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

1. Clone the Repository

git clone https://github.com/MDalamin5/End-to-End-Agentic-Ai-Automation-Lab.git
cd End-to-End-Agentic-Ai-Automation-Lab

2. Set Up Virtual Environment

It is recommended to use `conda` or `venv` to manage dependencies.

python -m venv venv
source venv/bin/activate  # On Windows use: venv\Scripts\activate

3. Install Dependencies

Dependencies may vary per module. Navigate to the specific project folder and install the requirements:

cd 18-e2e-chatbot-mem0-tools-HITL-MCP-RAG
pip install -r requirements.txt

4. Environment Variables

Copy the `.env.example` file (if available in the module) to `.env` and add your API keys (OpenAI, Anthropic, HuggingFace, etc.):

OPENAI_API_KEY="your_api_key_here"
ANTHROPIC_API_KEY="your_api_key_here"
TAVILY_API_KEY="your_api_key_here"

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 293 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. # 🤖 End-to-End Agentic AI & Automation Lab
  3. **A comprehensive, production-grade repository for building, deploying, and managing intelligent AI agents, RAG pipelines, and automated workflows.**
  4. [![GitHub stars](https://img.shields.io/github/stars/MDalamin5/End-to-End-Agentic-Ai-Automation-Lab?style=social)](https://github.com/MDalamin5/End-to-End-Agentic-Ai-Automation-Lab/stargazers)
  5. [![GitHub forks](https://img.shields.io/github/forks/MDalamin5/End-to-End-Agentic-Ai-Automation-Lab?style=social)](https://github.com/MDalamin5/End-to-End-Agentic-Ai-Automation-Lab/network/members)
  6. [![Python Version](https://img.shields.io/badge/Python-3.10%2B-blue?logo=python&logoColor=white)](https://www.python.org/)
  7. [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
  8. [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/)
  9. [Overview](#-overview) •
  10. [Key Highlights](#-key-highlights) •
  11. [Agent Harness](#-what-is-an-agent-harness) •
  12. [Project Architecture](#-repository-modules--projects) •
  13. [Tech Stack](#️-tech-stack--tools) •
  14. [Getting Started](#-getting-started)
  15. </div>
  16. ---
  17. ## 📖 Overview
  18. Welcome to the **End-to-End Agentic AI Automation Lab**. This repository is a massive, hands-on engineering playbook demonstrating how to transition from basic LLM API calls to **complex, multi-agent autonomous systems** and **production-ready AI products**.
  19. Whether you are looking to build highly reliable Agentic workflows using **LangGraph**, orchestrate multi-agent collaboration via **AutoGen**, implement cutting-edge **Model Context Protocol (MCP)**, or serve fine-tuned local models using **vLLM** and **Unsloth**, this repository has you covered.
  20. ---
  21. ## 🚀 Key Highlights
  22. * **Advanced Agentic Frameworks**: Deep dives into **LangGraph** (StateGraphs, subgraphs, memory, HITL) and **AutoGen** (RoundRobin, Swarm, custom tools).
  23. * **Model Context Protocol (MCP)**: Industry-grade implementations of Anthropic's MCP for tool execution, web search, and Notion integration.
  24. * **Production RAG Systems**: Implementation of Hybrid Search, BM25, LlamaParse, Semantic Routing, and Long/Short-Term Memory (Mem0).
  25. * **AI Workflow Automation**: Zero-code/low-code multi-agent orchestration using **n8n** and **LangFlow**.
  26. * **LLM Fine-Tuning & Serving**: Hands-on pipelines for fine-tuning with **LoRA/Unsloth** and deploying high-throughput inference endpoints with **vLLM**.
  27. * **End-to-End Products**: Complete full-stack implementations of an AI Interviewer, a Production ATS, and SynapseAI (a stateful, persistent chatbot).
  28. ---
  29. ## 🧰 What is an Agent Harness?
  30. An LLM on its own can only take text in and send text out. The **agent harness** is everything you build *around* the model so it can work in the real world and finish multi-step tasks. A simple way to put it: **Agent = Model + Harness**.
  31. ### What's in a harness
  32. | Component | Role |
  33. |---|---|
  34. | **System prompt** | Sets the agent's role, rules, and how it should behave |
  35. | **Tools** | Functions the model can call: search, SQL, code execution, APIs, MCP servers |
  36. | **Context management** | Picks what goes into the context window: summarization, context offloading, prompt caching |
  37. | **Memory** | Short-term (the conversation/state) and long-term (facts kept across sessions) |
  38. | **Filesystem / sandbox** | A place to write notes, save large results, and run code safely |
  39. | **Planning** | A to-do list that breaks a big task into steps |
  40. | **Subagents** | Specialized helper agents the main agent hands work to, each with its own clean context |
  41. | **Skills** | Reusable instructions and resources loaded only when they're needed |
  42. | **Human-in-the-loop (HITL)** | Pauses so a human can approve, edit, or reject sensitive actions |
  43. | **Middleware / hooks** | Code that runs before or after the model call to change inputs, outputs, or tool calls |
  44. **The job of a harness:** give the model the *right context at the right time* for the task in front of it.
  45. ### Why you need one
  46. * Lets the agent **take actions** in an environment, not just answer questions.
  47. * Connects the agent to **your data** and systems.
  48. * Keeps a **growing context** under control over long tasks.
  49. * Supports **HITL** approval for risky steps.
  50. * Runs work **in parallel** through subagents.
  51. * Helps the agent **improve over time** through memory and feedback.
  52. ### How the agent loop works
  53. ```
  54. ┌──────────────────────────────────────────┐
  55. User ───► │ Harness builds context │
  56. task │ (system prompt + memory + history + │
  57. │ available tools/skills) │
  58. └───────────────┬──────────────────────────┘
  59. ▼
  60. ┌───────────────────┐
  61. │ LLM (reasoning) │◄──────────────┐
  62. └─────────┬─────────┘ │
  63. final answer? │ tool call? │
  64. ┌─────────────┴─────────────┐ │
  65. ▼ ▼ │
  66. Return to user Harness runs the tool │
  67. (middleware, HITL │
  68. checks, sandbox) │
  69. │ │
  70. └─ result ─┘
  71. added back to context
  72. ```
  73. 1. **Input** – The user sends a task. The harness builds the prompt from the system prompt, memory, conversation history, and the tool definitions.
  74. 2. **Reason** – The model decides what to do: answer right away, call a tool, write a plan, or hand work to a subagent.
  75. 3. **Act** – The harness runs the tool call. Middleware can check or change it first, and HITL can pause for approval.
  76. 4. **Observe** – The tool result goes back into the context. Large outputs can be saved to the filesystem so the context stays small.
  77. 5. **Repeat** – The loop keeps going until the model gives a final answer or hits a stop condition.
  78. ### Working with an agent harness (Deep Agents example)
  79. [`deepagents`](https://github.com/langchain-ai/deepagents) is a ready-made harness built on LangGraph. It already includes planning, a virtual filesystem, subagents, and summarization, so you only add your model, prompt, and tools:
  80. ```python
  81. from deepagents import create_deep_agent
  82. def get_sum(a: int, b: int) -> int:
  83. """Return the sum of two numbers."""
  84. return a + b
  85. agent = create_deep_agent(
  86. model=model, # any LangChain chat model
  87. system_prompt="You are a helpful math assistant.",
  88. tools=[get_sum], # your custom tools
  89. # subagents=[...], middleware=[...], interrupt_on={...} # optional
  90. )
  91. result = agent.invoke(
  92. {"messages": [{"role": "user", "content": "What is the sum of 5 and 9?"}]}
  93. )
  94. print(result["messages"][-1].content)
  95. ```
  96. **Tips for working with agents**
  97. * **Write a clear system prompt.** Spell out the goal, the rules, and what output you expect.
  98. * **Keep tools small and well documented.** The model picks tools based on their names, docstrings, and type hints.
  99. * **Give the agent only the tools it needs.** Too many tools make it worse at choosing.
  100. * **Use subagents for big or separate subtasks** so the main agent's context stays clean.
  101. * **Add HITL for irreversible actions** such as sending emails, writing to a database, or making payments.
  102. * **Watch and trace runs** (for example with LangSmith) to debug the agent's decisions.
  103. 👉 Hands-on code: [`28-Intorduction-to-deep-and-Agents-Harness`](28-Intorduction-to-deep-and-Agents-Harness/)
  104. ---
  105. ## 📂 Repository Modules & Projects
  106. The lab is structured progressively. Click to expand each module to see the underlying projects:
  107. <details>
  108. <summary><b>1️⃣ Foundations & Data Ingestion (Modules 01 - 02)</b></summary>
  109. <br>
  110. * `01-Pydantic-Data-Validation`: Data structuring, field validation, and structured LLM outputs.
  111. * `02-LangChain-Basics`: Embedding models, VectorDBs (FAISS, Pinecone), and basic Retrieval-Augmented Generation (RAG) scratchpads.
  112. </details>
  113. <details>
  114. <summary><b>2️⃣ LangGraph & Workflow Orchestration (Modules 03 - 04, 13 - 14)</b></summary>
  115. <br>
  116. * `03-LangGraph-Introduction`: StateGraphs, Agentic workstations, multi-tool calling.
  117. * `04-LangGraph-Agentic-Workflows`: Agentic RAG, Multi-Agent Supervisors, Human-in-the-Loop (HITL), and Corrective RAG (CRAG).
  118. * `13-e2e-Deep-Agents`: Observation, evaluation, and reliable LangGraph applications.
  119. * `14-e2e-Ambient-Agent`: Building background-running autonomous agents.
  120. </details>
  121. <details>
  122. <summary><b>3️⃣ AutoGen Multi-Agent Systems (Modules 05 - 09)</b></summary>
  123. <br>
  124. * `05-Autogen-Introduction`: Async capabilities, tools, and basic teams.
  125. * `06-Autogen-HITL-and-Agentic-Orchestrator`: Selector Group Chats, Docker code execution, and Graph-based AutoGen.
  126. * `07-End-To-End-Projects-Autogen`: GPT Analyzer (Modular architecture), AI Interviewer.
  127. * `08-Advanced-Autogen-Team`: Swarm logic and Society of Mind teams.
  128. * `09-Autogen-RAG-and-Memory`: Integrating `mem0` for cross-session AutoGen memory.
  129. </details>
  130. <details>
  131. <summary><b>4️⃣ Model Context Protocol (MCP) & n8n (Modules 10 - 12)</b></summary>
  132. <br>
  133. * `10-MCP-All-You-Need`: Bridging AutoGen and LangChain with MCP. Lead collector, FireCrawl MCP, and Playwright MCP.
  134. * `11-MCP-based-End-to-End-Products`: Building fast, robust API backends utilizing MCP architectures via ngrok and FastAPI.
  135. * `12-n8n`: High-level automations. Chain of Agents, Social Media Content Generation, parallel agent logic, and Telegram bot integrations.
  136. </details>
  137. <details>
  138. <summary><b>5️⃣ Production RAG & Guardrails (Modules 17, 19)</b></summary>
  139. <br>
  140. * `17-Guardrails-for-llm`: Implementing NeMo Guardrails for secure and constrained LLM outputs.
  141. * `19-Productions-RAG`: Industry-practice RAG including `LlamaParse`, BM25/Hybrid Search, HyDE, chunking strategies, and Reranking pipelines.
  142. </details>
  143. <details>
  144. <summary><b>6️⃣ LLM Fine-Tuning & Deployment (Modules 21 - 22)</b></summary>
  145. <br>
  146. * `21-LLM-Deployment-vLLM`: Deploying models for high-throughput generation using vLLM and accessing via LangChain SDK.
  147. * `22-LLM-FineTune-Deployment`: Model fine-tuning using **Unsloth**, **LoRA**, HuggingFace Pipelines, and quantization setups for edge devices.
  148. </details>
  149. <details>
  150. <summary><b>7️⃣ End-to-End Full-Stack Projects (Modules 18, 20, 23, 24)</b></summary>
  151. <br>
  152. * `18-e2e-chatbot-mem0-tools-HITL-MCP-RAG`: A massive implementation of a fully-featured chatbot with long/short-term memory, PostgreSQL persistence, and streaming UI.
  153. * `20-e2e-Productions-grade-ATS`: End-to-end Applicant Tracking System backed by Alembic, SQLModel, and LangGraph.
  154. * `23-e2e-multi-agent-plan-research-write-blog`: A multi-agent writer architecture with a beautiful web frontend.
  155. * `24-SynapseAI-parsitence-chatbot`: A modern API-first chatbot backend via FastAPI with complex graph routing.
  156. </details>
  157. <details>
  158. <summary><b>8️⃣ Deep Agents & Agent Harness (Module 28)</b></summary>
  159. <br>
  160. * `28-Intorduction-to-deep-and-Agents-Harness`: What an agent harness is, Deep Agents architecture (execution environment, context management, delegation, steering), middleware, system prompts, prompt caching, and search/SQL agents built with `deepagents`.
  161. </details>
  162. ---
  163. ## 🛠️ Tech Stack & Tools
  164. **Core AI/ML:**
  165. ![PyTorch](https://img.shields.io/badge/PyTorch-%23EE4C2C.svg?style=for-the-badge&logo=PyTorch&logoColor=white)
  166. ![LangChain](https://img.shields.io/badge/🦜_LangChain-02569B?style=for-the-badge&logoColor=white)
  167. ![vLLM](https://img.shields.io/badge/vLLM-000000?style=for-the-badge&logo=v&logoColor=white)
  168. ![HuggingFace](https://img.shields.io/badge/%F0%9F%A4%97_HuggingFace-FFD21E?style=for-the-badge&logoColor=black)
  169. **Agentic & Orchestration:**
  170. ![LangGraph](https://img.shields.io/badge/🕸️_LangGraph-1C3C3C?style=for-the-badge&logoColor=white)
  171. ![AutoGen](https://img.shields.io/badge/🤖_AutoGen-4B0082?style=for-the-badge&logoColor=white)
  172. ![n8n](https://img.shields.io/badge/n8n-%23EA4B3D.svg?style=for-the-badge&logo=n8n&logoColor=white)
  173. ![MCP](https://img.shields.io/badge/MCP_(Anthropic)-5B0000?style=for-the-badge)
  174. **Backend & Data:**
  175. ![FastAPI](https://img.shields.io/badge/FastAPI-005571?style=for-the-badge&logo=fastapi)
  176. ![Postgres](https://img.shields.io/badge/postgres-%23316192.svg?style=for-the-badge&logo=postgresql&logoColor=white)
  177. ![Redis](https://img.shields.io/badge/redis-%23DD0031.svg?style=for-the-badge&logo=redis&logoColor=white)
  178. ![Docker](https://img.shields.io/badge/docker-%230db7ed.svg?style=for-the-badge&logo=docker&logoColor=white)
  179. ---
  180. ## ⚙️ Getting Started
  181. ### 1. Clone the Repository
  182. ```bash
  183. git clone https://github.com/MDalamin5/End-to-End-Agentic-Ai-Automation-Lab.git
  184. cd End-to-End-Agentic-Ai-Automation-Lab
  185. ```
  186. ### 2. Set Up Virtual Environment
  187. It is recommended to use `conda` or `venv` to manage dependencies.
  188. ```bash
  189. python -m venv venv
  190. source venv/bin/activate # On Windows use: venv\Scripts\activate
  191. ```
  192. ### 3. Install Dependencies
  193. Dependencies may vary per module. Navigate to the specific project folder and install the requirements:
  194. ```bash
  195. cd 18-e2e-chatbot-mem0-tools-HITL-MCP-RAG
  196. pip install -r requirements.txt
  197. ```
  198. ### 4. Environment Variables
  199. Copy the `.env.example` file (if available in the module) to `.env` and add your API keys (OpenAI, Anthropic, HuggingFace, etc.):
  200. ```env
  201. OPENAI_API_KEY="your_api_key_here"
  202. ANTHROPIC_API_KEY="your_api_key_here"
  203. TAVILY_API_KEY="your_api_key_here"
  204. ```
  205. ---
  206. ## 🤝 Contributing
  207. This repository is continuously evolving! Contributions, bug reports, and feature requests are highly welcome.
  208. 1. Fork the Project
  209. 2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
  210. 3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)
  211. 4. Push to the Branch (`git push origin feature/AmazingFeature`)
  212. 5. Open a Pull Request
  213. ---
  214. ## 📜 License & Connect
  215. Distributed under the **MIT License**. See [`LICENSE`](LICENSE) for more information.
  216. <div align="center">
  217. **Developed with 💡 by [Md Al Amin](https://github.com/MDalamin5)**
  218. [![LinkedIn](https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white)](https://linkedin.com/in/mdalamin5/)
  219. [![GitHub](https://img.shields.io/badge/GitHub-100000?style=for-the-badge&logo=github&logoColor=white)](https://github.com/MDalamin5)
  220. If you find this repository helpful, don't forget to ⭐ star it!
  221. </div>

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.