Skip to content
Back to search

multimodal-agents-course

An MCP Multimodal AI Agent with eyes and ears!

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/the-ai-merge--multimodal-agents-course/badge.svg)](/listings/the-ai-merge--multimodal-agents-course)

HTML

<a href="/listings/the-ai-merge--multimodal-agents-course"><img src="/api/listings/the-ai-merge--multimodal-agents-course/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Kubrick is **not** a simple tutorial. So, to get this system up and running, there are a few things you need to do first.

We have detailed the steps to get you started in this [GETTING_STARTED.md](GETTING_STARTED.md) file.

> πŸ’‘ Having Kubrick running is just the first step! Now that you have it up and running, it's time to actually understand how it works (see [Course Syllabus](#-course-syllabus)).

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md Β· README Β· 195 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <h1 align="center">Kubrick Course</h1>
  2. <p align="center">
  3. <img alt="logo" src="static/hal_9000.png" width=100 />
  4. <h4 align="center" >Hi Dave...</h4>
  5. </p>
  6. <hr>
  7. <p align="center">
  8. <h4 align="center">Learn to build AI Agents that can understand images, text, audio and videos.</h3>
  9. </p>
  10. <p align="center"> A <b>free, Open-source</b> course by <a href="https://theneuralmaze.substack.com"> The Neural Maze </a> and <a href="https://neuralbits.substack.com">Neural Bits</a> in collaboration with <a href="https://github.com/pixeltable">Pixeltable</a> and <a href="https://github.com/comet-ml/opik">Opik</a>
  11. </p>
  12. </br>
  13. <img alt="logo" src="kubrick-api/static/agent_architecture.gif" width=1000 />
  14. </br>
  15. ---
  16. ## πŸ“– About This Course
  17. Tired of tutorials that just walk you through connecting an existing MCP server to Claude Desktop?
  18. Yeah, us too.
  19. That's why we built **Kubrick AI**, an MCP Multimodal Agent for video processing tasks. Yes! You read that right.
  20. > πŸ’‘ Agents + Video Processing ... and MCP!
  21. This course is a collaboration between The Neural Maze and Neural Bits (from now on, "The Neural Bros"), and it's built for developers who want to go beyond the basics and build serious, production-ready AI Systems. In particular, you'll:
  22. * Learn how to build an MCP server for video processing using Pixeltable and FastMCP
  23. * Design a custom, Groq-powered agent, connected to your MCP server with its own MCP client
  24. * Integrate your agentic system with Opik for full observability and prompt versioning
  25. ## πŸ–ŠοΈ What you'll learn
  26. * Learn how to use Pixeltable for multimodal data processing and stateful agents
  27. * Create complex MCP servers using FastMCP: expose resources, prompts, and tools
  28. * Apply prompt versioning to your MCP server (instead of defining the prompts in the Agent API)
  29. * Learn how to implement custom MCP clients for your agents
  30. * Implement an MCP Tool Agent from scratch, using Llama 4 Scout and Maverick as the LLMs
  31. * Use Opik for MCP prompt versioning
  32. * Learn how to implement custom tracing and monitoring with Opik
  33. > πŸš€ No shortcuts. No fluff. Let's learn by doing.
  34. ---
  35. ## πŸ’» What You'll Do:
  36. Completing this course, you'll learn how to design and enable Agents to understand multimodal data, across images, video, audio, and text inputs, all within a single system.
  37. Specifically, you'll get to:
  38. - Build a complex Multimodal Processing Pipeline
  39. - Build a Video Search Engine and expose its functionality to an Agent via MCP (Model Context Protocol)
  40. - Build a production-ready API to power the Agent.
  41. - Integrate LLMOps principles and best software engineering practices.
  42. - Learn about video, embeddings, streaming APIs, Vision Language Models (VLMs), and more.
  43. After completing this course, you'll have built your own Kubrick Agent with a HAL-themed spin-off, to play the role of a new set of eyes and ears:
  44. <video src="https://github.com/user-attachments/assets/ef77c2a9-1a77-4f14-b2dd-e759c3f6db72"/></video>
  45. ---
  46. ## Getting Started
  47. Kubrick is **not** a simple tutorial. So, to get this system up and running, there are a few things you need to do first.
  48. We have detailed the steps to get you started in this [GETTING_STARTED.md](GETTING_STARTED.md) file.
  49. > πŸ’‘ Having Kubrick running is just the first step! Now that you have it up and running, it's time to actually understand how it works (see [Course Syllabus](#-course-syllabus)).
  50. ---
  51. ## Watch the Full Video Course
  52. <p align="center">
  53. <a href="https://www.youtube.com/watch?v=_iYB1z1_Xgs&t=316s"><img src="static/video_thumbnail.png" alt="Kubrick Multimodal Agent" width="500"></a>
  54. </p>
  55. ---
  56. ## πŸ§‘β€πŸŽ“ Who is this course for?
  57. You'll get the most out of this course by building it yourself, from the ground up.
  58. The course components are structured to cover key concepts and demonstrate how to build upon them, ultimately leading to AI Systems.
  59. | Target Audience | Skills you'll get |
  60. | ----- | ------------------|
  61. | ML/AI Engineers | Build complex MCP Servers, learn to apply AI Models to Video, Images, and Speech.|
  62. | Software Engineers | Learn to connect AI Components with APIs, building end-to-end agentic applications.|
  63. | Data Engineers/Scientists | Learn to design an AI System, managing Video/Audio/Image data processing and structure.
  64. Regardless of your experience or title, this course aims to unpack complex topics in practical terms and concepts you could understand, learn, and apply - helping you to build a complete AI system.
  65. ## πŸŽ“ Prerequisites
  66. In this section, we outlined a few requirements and nice-to-haves to improve your learning experience while taking this course.
  67. | Category | Label | Description |
  68. |----------| ----- | ----------- |
  69. | Programming Skills (Beginner) | Requirement | Understanding of Programming in general, and the Python language syntax.
  70. | AI/ML Concepts (Beginner) | Nice to Have | Understanding the basic concepts behind AI, AI Models, and AI Systems.
  71. | LLMs, MCP, Agents | Nice to Have | Perfect if you know about them, not a problem if you don't. We'll teach and explain it step by step.
  72. | Laptop/PC with any OS | Requirement | AI Models inference requires compute. To overcome that, we'll mainly use API based models.|
  73. The overall level of this course is Beginner/Intermediate, but don't worry. We'll aim to explain every component step by step designed for a larger audience.
  74. ## πŸ’° What's the cost?
  75. This course and its materials are open-source and completely free, thanks to our sponsors, Pixeltable and Opik!
  76. You'll be able to run Kubrick examples while staying at a minimum cost. That's because we'll use OpenAI and Groq for our LLM and VLM calls, which offer freemium plans as such:
  77. | Provider | Free Credits |
  78. | -------- | ----------- |
  79. | OpenAI | $5 on first sign-up |
  80. | Groq | 500,000 tokens/day
  81. > [!NOTE]
  82. > In this setup, for running the Kubrick Agent example, the freemium plans are enough.
  83. ## πŸ“š Course Syllabus
  84. The Kubrick Agent open-source course consists of five comprehensive modules, covering concepts, system design, tooling, and hands-on implementation.
  85. To get the most out of this course, we recommend:
  86. - Clone this repository.
  87. - Follow the Get Started Sections of each module.
  88. - Read the module articles to understand each component.
  89. - Follow the Video Lessons for hands-on implementation.
  90. - Set up the code and run the project.
  91. | Module No. | In-depth Lesson (Link) | Lesson Summary (3min) | Description | Code |
  92. | - | -------------- | ------------ | -------- | ----------- |
  93. | 0 | <a href="https://multimodalai.substack.com/p/introducing-kubrick-course-a-multimodal"><img src="static/kubrick_ai_diagram.png" width=300></a> | [Get a Summary](https://theneuralmaze.substack.com/p/your-first-video-agent-multimodality) | Course introduction and overview. Outlining the basic components | N/A
  94. | 1 | <a href="https://multimodalai.substack.com/p/cracking-the-code-of-multimodal-ai"><img src="kubrick-mcp/static/m2_mm_data_pipelines.png" width=300></a> | [Get a Summary](https://theneuralmaze.substack.com/p/the-future-of-video-search-is-here) | Diving into the core components of the multimodal processing pipeline, covering video, images, text, and audio data. | [kubrick-mcp](kubrick-mcp)
  95. | 2 | <a href="https://multimodalai.substack.com/p/building-mcp-servers-for-video-search"><img src="kubrick-mcp/static/m2_mcp_servers_for_mm.png" width=300></a> | [Get a Summary](https://theneuralmaze.substack.com/p/how-real-ai-engineers-build-mcp-servers)| Building the Kubrick MCP Server with FastMCP. Covering MCP protocol (transport, communication), capabilities, tools, resources, and prompts. Adding Opik Integration and MCP Inspector for efficient debugging and local testing. | [kubrick-mcp](kubrick-mcp)
  96. | 3 | <a href="https://multimodalai.substack.com/p/building-tool-use-mcp-agents-from"><img src="kubrick-api/static/m3_tooluse_agents.png" width=300></a> | [Get a Summary](https://theneuralmaze.substack.com/p/how-we-built-an-mcp-agent-from-scratch) | Building the Kubrick MCP Agent from scratch. Memory layer implementation with Pixeltable and MCP Client with FastMCP. Understanding how to translate MCP Tools into specific provider tools (Groq) | [kubrick-api](kubrick-api)
  97. | 4 | <a href="https://multimodalai.substack.com/p/building-a-fullstack-befe-agent-ui"><img src="kubrick-api/static/m4_building_agent_api.png" width=300></a> | [Get a Summary](https://theneuralmaze.substack.com/p/if-you-cant-trace-it-monitor-it-or) | Building the Groq Agent, covering the React UI - FastAPI API communication and adding LLMOps observability and conversation tracing with Opik (by Comet) | [kubrick-api](kubrick-api)
  98. | 5 | <a href="https://multimodalai.substack.com/p/kubrick-course-final-round-up"><img src="kubrick-api/static/agent_architecture.gif" width=300></a> | [Get a Summary](https://theneuralmaze.substack.com/p/the-ai-engineering-course-i-wish) | Complete Round-up of the free, open-source Kubrick Course. | N/A
  99. ## Sponsors
  100. | | |
  101. |:---:|:---:|
  102. | <a href="https://www.pixeltable.com/" target="_blank"><img src="static/sponsors/pixeltable.png" width="250" alt="Pixeltable"></a> | <a href="https://github.com/comet-ml/opik" target="_blank"> <img src="static/sponsors/opik.png" width="250" alt="Opik"></a>|
  103. | <p align="center">**Pixeltable** </br> is the only Python framework that provides incremental storage, transformation, indexing, and orchestration of your multimodal data.</p> | <p align="center">**Opik** </br>LLM Evaluation platform helps you build, evaluate, and optimize LLM systems that run better, faster, and cheaper</p>
  104. ## Authors
  105. <div align="center">
  106. <table>
  107. <tr>
  108. <td align="center">
  109. <a href="https://github.com/MichaelisTrofficus">
  110. <img src="https://github.com/MichaelisTrofficus.png" width="100px;" alt="Miguel Otero Pedrido"/><br />
  111. <sub><b>Miguel Otero Pedrido</b></sub>
  112. </a><br />
  113. <sub>AI / ML Engineer</sub>
  114. <p>
  115. </br>
  116. <a href="https://linkedin.com/in/migueloteropedrido" target="_blank">
  117. <img src="https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white" alt="LinkedIn">
  118. </a>
  119. </p>
  120. </td>
  121. <td align="center">
  122. <a href="https://github.com/arazvant">
  123. <img src="https://github.com/arazvant.png" width="100px;" alt="Alex Razvant"/><br />
  124. <sub><b>Alex Razvant</b></sub>
  125. </a><br />
  126. <sub>AI / ML Engineer</sub>
  127. <p>
  128. </br>
  129. <a href="https://www.linkedin.com/in/arazvant" target="_blank">
  130. <img src="https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white" alt="LinkedIn">
  131. </a>
  132. </p>
  133. </td>
  134. </tr>
  135. </table>
  136. </div>
  137. ## Find us on
  138. | Publication | Description | Subscribe |
  139. |:---:|---|:---:|
  140. | <a href="https://neuralbits.substack.com/" aria-label="Neural Bits"><img src="https://avatars.githubusercontent.com/u/114508542?s=400&u=abae5b0d28f288fc12502b170167a2ccce133f31&v=4" alt="Neural Bits Logo" width="100"/><br/><p align="center">**Neural Bits**</a></p> | Learn about industry standards and production-ready AI Engineering. | <a href="https://neuralbits.substack.com/"><img src="https://img.shields.io/static/v1?label&logo=substack&message=Subscribe%20Now&style=for-the-badge&color=black&scale=2" alt="Subscribe Now" height="30"></a> |
  141. | <a href="https://theneuralmaze.substack.com/" aria-label="The Neural Maze"><img src="https://avatars.githubusercontent.com/u/151655127?s=400&u=2fff53e8c195ac155e5c8ee65c6ba683a72e655f&v=4" alt="The Neural Maze Logo" width="100"/><br/><p align="center">**The Neural Maze**</a> </a> | Learn to build AI Systems that actually work, from principles to production. Delivered every Wednesday. | <a href="https://theneuralmaze.substack.com/"><img src="https://img.shields.io/static/v1?label&logo=substack&message=Subscribe%20Now&style=for-the-badge&color=black&scale=2" alt="Subscribe Now" height="30"></a> |

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.