Skip to content
Back to search

ai-gateway

Labs to explore AI Models, MCP servers, and Agents with the AI Gateway powered by Azure API Management and Microsoft Foundry πŸš€

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/azure-samples/ai-gateway with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/azure-samples/ai-gateway'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/azure-samples--ai-gateway/badge.svg)](/listings/azure-samples--ai-gateway)

HTML

<a href="/listings/azure-samples--ai-gateway"><img src="/api/listings/azure-samples--ai-gateway/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

These steps run a script from the network that findsafeskills does not scan.

Prerequisites

- [Python 3.12+](https://www.python.org/) - [uv](https://docs.astral.sh/uv/) (fast Python package manager) β€” install via `curl -LsSf https://astral.sh/uv/install.sh | sh` (Linux/macOS) or `powershell -c "irm https://astral.sh/uv/install.ps1 | iex"` (Windows) - [VS Code](https://code.visualstudio.com/) with [Jupyter extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) - [Azure Subscription](https://azure.microsoft.com/free/) with Contributor + RBAC Administrator roles - [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli) authenticated to your subscription

Get Started

# Clone the repository
git clone https://github.com/Azure-Samples/AI-Gateway.git
cd AI-Gateway

# Create the virtual environment and install dependencies
uv sync
uv pip install -r pyproject.toml

# Open VS Code and start with a lab
code .

When opening a notebook, select the `.venv` interpreter created by `uv sync` as the Jupyter kernel.

Or launch instantly with **[GitHub Codespaces](https://codespaces.new/Azure-Samples/AI-Gateway/tree/main)** ☁️

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md Β· README Β· 209 lines Β· 2 findings marked

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <!-- markdownlint-disable MD033 -->
  2. <div align="center">
  3. # ✨ AI Gateway Labs
  4. [![Open Source](https://img.shields.io/badge/Open%20Source-❀️-blue)](https://github.com/Azure-Samples/AI-Gateway)
  5. [![GitHub Stars](https://img.shields.io/github/stars/Azure-Samples/AI-Gateway?style=social)](https://github.com/Azure-Samples/AI-Gateway/stargazers)
  6. [![Open in GitHub Codespaces](https://img.shields.io/badge/Open%20in-Codespaces-orange?logo=github)](https://codespaces.new/Azure-Samples/AI-Gateway)
  7. [![AI-Gateway Labs](images/ai-gateway-labs-banner.png)](http://aka.ms/ai-gateway/labs)
  8. ### Explore the enterprise-grade gateway for managing AI Models, Tools, and Agents
  9. <br/>
  10. [![AI-Gateway flow](images/ai-gateway.gif)](http://aka.ms/ai-gateway/labs)
  11. [![Azure](https://img.shields.io/badge/Powered%20by-Azure%20API%20Management-0078D4)](https://learn.microsoft.com/azure/api-management/genai-gateway-capabilities)
  12. </div>
  13. > πŸ“° **New!** The [AI Gateway Dev Portal](https://aka.ms/ai-gateway/dev-portal) is now live β€” A starting point for building your own developer portal on top of Azure API Management AI Gateway's. Fork it, open it in VS Code with GitHub Copilot (or any coding agent), and shape it to fit your needs!
  14. ## Why AI Gateway?
  15. Building production-ready AI applications requires more than just calling model APIs. You need **security**, **reliability**, **observability**, and **cost control**β€”without slowing down innovation.
  16. [**AI Gateway**](https://learn.microsoft.com/azure/api-management/genai-gateway-capabilities) powered by [Azure API Management](https://learn.microsoft.com/en-us/azure/api-management/) provides:
  17. - πŸ” **Security** β€” OAuth 2.0, managed identities, content safety filtering
  18. - ⚑ **Performance** β€” Load balancing, semantic caching, request routing
  19. - πŸ“Š **Observability** β€” Token metrics, built-in logging, tracing
  20. - πŸ’° **Cost Control** β€” Rate limiting, quota management, FinOps framework
  21. - πŸ”Œ **Extensibility** β€” MCP protocol support, function calling, multi-model routing
  22. ## πŸ“š Explore the Labs
  23. > **πŸ”— Browse all 30+ labs at [aka.ms/ai-gateway/labs](http://aka.ms/ai-gateway/labs)**
  24. Each lab is a hands-on Jupyter notebook with step-by-step instructions, Bicep infrastructure templates, and APIM policies you can deploy to your Azure subscription.
  25. ## 🧠 AI Gateway for Models
  26. Manage and control access to Large Language Models with enterprise-grade policies.
  27. | Lab | Description |
  28. |-----|-------------|
  29. | [**Backend Pool Load Balancing**](labs/backend-pool-load-balancing/backend-pool-load-balancing.ipynb) | Distribute requests across multiple model endpoints |
  30. | [**Token Rate Limiting**](labs/token-rate-limiting/token-rate-limiting.ipynb) | Control token consumption with rate limiting policies |
  31. | [**Semantic Caching**](labs/semantic-caching/semantic-caching.ipynb) | Cache responses using vector similarity for faster, cheaper completions |
  32. | [**Model Routing**](labs/model-routing/model-routing.ipynb) | Route requests to different backends based on model and version |
  33. | [**FinOps Framework**](labs/finops-framework/finops-framework.ipynb) | Manage AI budgets with automated quota controls |
  34. ## πŸ”§ AI Gateway for Tools
  35. Enable secure tool access with MCP protocol and function calling capabilities.
  36. | Lab | Description |
  37. |-----|-------------|
  38. | [**Model Context Protocol (MCP)**](labs/model-context-protocol/model-context-protocol.ipynb) | Plug & play tools with OAuth credential management |
  39. | [**MCP Client Authorization**](labs/mcp-client-authorization/mcp-client-authorization.ipynb) | Implement MCP with the client authorization flow |
  40. | [**Function Calling**](labs/function-calling/function-calling.ipynb) | Use OpenAI function calling with Azure Functions backend |
  41. | [**Realtime Audio + MCP**](labs/realtime-mcp-agents/realtime-mcp-agents.ipynb) | Combine realtime voice API with MCP tools |
  42. ## πŸ€– AI Gateway for Agents
  43. Build and control agentic applications with orchestration frameworks.
  44. | Lab | Description |
  45. |-----|-------------|
  46. | [**AI Agent Service**](labs/ai-agent-service/ai-agent-service-v2.ipynb) | Explore Foundry Agent Service with multi-service control |
  47. | [**OpenAI Agents SDK**](labs/openai-agents/openai-agents.ipynb) | Use OpenAI Agents with Azure OpenAI and APIM-managed tools |
  48. | [**Gemini MCP Agents**](labs/gemini-mcp-agents/gemini-mcp-agents.ipynb) | Integrate Google Gemini models with MCP tools |
  49. | [**A2A Enabled Agents**](labs/mcp-a2a-agents/mcp-agent-as-a2a-server.ipynb) | A2A-enabled Agents with models and MCP plug & play tools |
  50. ## πŸš€ Quick Start
  51. ### Prerequisites
  52. - [Python 3.12+](https://www.python.org/)
  53. - [uv](https://docs.astral.sh/uv/) (fast Python package manager) β€” install via `curl -LsSf https://astral.sh/uv/install.sh | sh` (Linux/macOS) or `powershell -c "irm https://astral.sh/uv/install.ps1 | iex"` (Windows)
    • Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
    • Note: Runs a downloaded PowerShell script (iex / Invoke-Expression)
    • Line 82 on GitHub
  54. - [VS Code](https://code.visualstudio.com/) with [Jupyter extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter)
  55. - [Azure Subscription](https://azure.microsoft.com/free/) with Contributor + RBAC Administrator roles
  56. - [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli) authenticated to your subscription
  57. ### Get Started
  58. ```bash
  59. # Clone the repository
  60. git clone https://github.com/Azure-Samples/AI-Gateway.git
  61. cd AI-Gateway
  62. # Create the virtual environment and install dependencies
  63. uv sync
  64. uv pip install -r pyproject.toml
  65. # Open VS Code and start with a lab
  66. code .
  67. ```
  68. When opening a notebook, select the `.venv` interpreter created by `uv sync` as the Jupyter kernel.
  69. Or launch instantly with **[GitHub Codespaces](https://codespaces.new/Azure-Samples/AI-Gateway/tree/main)** ☁️
  70. ## πŸ”¨ Developer Tools
  71. The [`tools/`](tools/) folder provides utilities for testing and development:
  72. | Tool | Description |
  73. |------|-------------|
  74. | [**Tracing**](tools/tracing.ipynb) | Invoke AI Foundry APIs with tracing enabled |
  75. | [**Streaming**](tools/streaming.ipynb) | Test streaming responses from AI models |
  76. | [**Rate Limit Tester**](tools/rate-limit.ipynb) | Validate rate limiting configurations |
  77. | [**Mock Server**](tools/mock-server/mock-server.ipynb) | OpenAI API mock for local development and testing |
  78. | [**OAuth Client**](tools/client-oauth.ipynb) | Test OAuth authentication flows |
  79. ## πŸ‘©β€πŸ’» Build Your Own Labs with AI
  80. This repository includes **Copilot Agent Skills** that help you create new labs using AI-assisted development in VS Code.
  81. ### Available Skills
  82. | Skill | Description |
  83. |-------|-------------|
  84. | `lab-creator` | Scaffolds new labs with notebooks, Bicep, and policies |
  85. | `apim-bicep` | Generates Azure Bicep templates for APIM resources |
  86. | `apim-terraform` | Generates Terraform configurations for APIM |
  87. | `apim-policies` | Creates APIM XML policies for AI gateway scenarios |
  88. | `apim-kql` | Generates queries in KQL to control models, tools and agents |
  89. | `mcp-builder` | Builds MCP servers for tool integration |
  90. ### Example: Create a New Lab
  91. Open this repo in VS Code with GitHub Copilot and use this prompt:
  92. ```
  93. Create a new lab called "multi-model-failover" that demonstrates how to
  94. implement automatic failover between different AI models when the primary
  95. model is unavailable or throttled. Include:
  96. - A backend pool with priority-based routing
  97. - Retry policy with exponential backoff
  98. - Circuit breaker pattern for unhealthy backends
  99. - Built-in LLM logging to track usage across all backends
  100. - Test the model with a LangChain agent: https://docs.langchain.com/oss/python/langchain/agents
  101. Use gpt-4.1-mini as primary and gpt-4.1-nano as fallback, deploy to Sweden Central.
  102. ```
  103. Copilot will generate the complete lab structure including:
  104. - πŸ““ Jupyter notebook with step-by-step instructions
  105. - 🦾 Bicep infrastructure template
  106. - βš™οΈ APIM policy XML
  107. - πŸ“– README documentation
  108. - 🧹 Cleanup notebook
  109. ## πŸ›οΈ Well-Architected Framework
  110. Labs are designed following [Azure Well-Architected Framework](https://learn.microsoft.com/azure/well-architected/what-is-well-architected-framework) principles:
  111. | Pillar | Labs |
  112. |--------|------|
  113. | **Security** | Access controlling, Content safety, Private connectivity |
  114. | **Reliability** | Backend pool load balancing, Token rate limiting |
  115. | **Performance** | Semantic caching, Model routing |
  116. | **Operations** | Built-in logging, Token metrics emitting |
  117. | **Cost** | FinOps framework, Semantic caching |
  118. ## πŸ“• Enterprise AI Gateway e-Book
  119. <img align="left" width="200" src="images/ebook.png" alt="Enterprise AI Gateway eBook">
  120. Download the <a href="docs/media/Enterprise%20AI%20Gateway%20eBook%20-%20Feb%202026.pdf">Enterprise AI Gateway e-Book</a> for comprehensive end-to-end view of the Enterprise AI Gateway pattern, explaining why a centralized governance layer is essential for organizations adopting AI at scale and how it can be practically implemented using Azure API Management and Microsoft Foundry.
  121. It describes the AI Gateway as a control plane that mediates all interactions between AI apps and agents and the underlying models, data, and tools, enabling consistent enforcement of security, safety, cost controls, resiliency, scalability, observability, and governance. Overall, the e-Book positions the Enterprise AI Gateway as a foundational architectural component that allows enterprises to innovate rapidly with AI while maintaining trust, compliance, visibility, and control.
  122. <br clear="left"/>
  123. ## 🎬 Conferences & Webcasts
  124. Learn from experts through these videos covering AI Gateway concepts and implementations.
  125. | | | | |
  126. |:---:|:---:|:---:|:---:|
  127. | [![Build 2025](https://i.ytimg.com/vi/TG4ZULpS81w/mqdefault.jpg)](https://www.youtube.com/watch?v=TG4ZULpS81w) | [![Reactor Jan 2025](https://i.ytimg.com/vi/yNkK6q1vFF8/mqdefault.jpg)](https://www.youtube.com/watch?v=yNkK6q1vFF8) | [![MCP Workflows](https://i.ytimg.com/vi/ian7K6QKgJQ/mqdefault.jpg)](https://www.youtube.com/watch?v=ian7K6QKgJQ) | [![Supercharge your API's](https://i.ytimg.com/vi/ou3QEAcxW3o/mqdefault.jpg)](https://www.youtube.com/watch?v=ou3QEAcxW3o)
  128. [![Ignite 2024](https://i.ytimg.com/vi/HuPNgrjMcuI/mqdefault.jpg)](https://www.youtube.com/watch?v=HuPNgrjMcuI) | [![Reactor Nov 2024](https://i.ytimg.com/vi/gOopye0Hwo4/mqdefault.jpg)](https://www.youtube.com/watch?v=gOopye0Hwo4) | [![Content Safety](https://i.ytimg.com/vi/WCtcBL4GZXs/mqdefault.jpg)](https://www.youtube.com/watch?v=WCtcBL4GZXs)| [![Semantic Caching](https://i.ytimg.com/vi/8Fzu8kF47Bw/mqdefault.jpg)](https://www.youtube.com/watch?v=8Fzu8kF47Bw)
  129. [![Token Emit Metric](https://i.ytimg.com/vi/2pW6Z2VwHmQ/mqdefault.jpg)](https://www.youtube.com/watch?v=2pW6Z2VwHmQ) | [![GenAI Gateway](https://i.ytimg.com/vi/dBTCgzTe900/mqdefault.jpg)](https://www.youtube.com/watch?v=dBTCgzTe900) | [![Control AI Services](https://i.ytimg.com/vi/yYt1-a7RocU/mqdefault.jpg)](https://www.youtube.com/watch?v=yYt1-a7RocU) | [![John Savill](https://i.ytimg.com/vi/l_8dTUwrqNw/mqdefault.jpg)](https://www.youtube.com/watch?v=l_8dTUwrqNw)
  130. [![Houssem Dellai](https://i.ytimg.com/vi/C17qTB3fL3Y/mqdefault.jpg)](https://www.youtube.com/watch?v=C17qTB3fL3Y) | [![Turbo360](https://i.ytimg.com/vi/QRCHKN9PX7o/mqdefault.jpg)](https://www.youtube.com/watch?v=QRCHKN9PX7o) | [![A2A MCP Multiagents](https://i.ytimg.com/vi/ylzdM1KangA/mqdefault.jpg)](https://www.youtube.com/watch?v=ylzdM1KangA) | | |
  131. ## πŸ“– Resources
  132. - πŸ“˜ [AI Gateway Documentation](https://learn.microsoft.com/azure/api-management/genai-gateway-capabilities)
  133. - πŸŽ“ [AI Gateway Workshop](https://aka.ms/ai-gateway/workshop)
  134. - βš™οΈ [APIM Samples](https://github.com/Azure-Samples/Apim-Samples)
  135. - πŸ“š [APIM Learning Modules](https://learn.microsoft.com/training/browse/?products=azure-api-management)
  136. - πŸ’‘ [Landing Zone Accelerator](https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/app-platform/api-management/landing-zone-accelerator)
  137. - πŸ›οΈ [Foundry Ctadel](https://aka.ms/foundry-citadel)
  138. - πŸ“° [API Management News](https://techcommunity.microsoft.com/tag/API%20Management)
  139. ## 🀝 Contributing
  140. Contributions are welcome! Please read [CONTRIBUTING.md](CONTRIBUTING.MD) for guidelines.
  141. ## License
  142. [MIT](LICENSE)
  143. <div align="center">
  144. [![APIM Love](https://img.shields.io/badge/APIM%20Love-❀️-blue)](http://aka.ms/apimlove)
  145. </div>

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.