Skip to content
Back to search

BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/bentoml/BentoML with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/bentoml/BentoML'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/bentoml--bentoml/badge.svg)](/listings/bentoml--bentoml)

HTML

<a href="/listings/bentoml--bentoml"><img src="/api/listings/bentoml--bentoml/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Install BentoML:

# Requires Python≥3.9
pip install -U bentoml

Define APIs in a `service.py` file.

import bentoml

@bentoml.service(
    image=bentoml.images.Image(python_version="3.11").python_packages("torch", "transformers"),
)
class Summarization:
    def __init__(self) -> None:
        import torch
        from transformers import pipeline

        device = "cuda" if torch.cuda.is_available() else "cpu"
        self.pipeline = pipeline('summarization', device=device)

    @bentoml.api(batchable=True)
    def summarize(self, texts: list[str]) -> list[str]:
        results = self.pipeline(texts)
        return [item['summary_text'] for item in results]

💻 Run locally

Install PyTorch and Transformers packages to your Python virtual environment.

pip install torch transformers  # additional dependencies for local run

Run the service code locally (serving at http://localhost:3000 by default):

bentoml serve

You should expect to see the following output.

[INFO] [cli] Starting production HTTP BentoServer from "service:Summarization" listening on http://localhost:3000 (Press CTRL+C to quit)
[INFO] [entry_service:Summarization:1] Service Summarization initialized

Now you can run inference from your browser at http://localhost:3000 or with a Python script:

import bentoml

with bentoml.SyncHTTPClient('http://localhost:3000') as client:
    summarized_text: str = client.summarize([bentoml.__doc__])[0]
    print(f"Result: {summarized_text}")

🐳 Deploy using Docker

Run `bentoml build` to package necessary code, models, dependency configs into a Bento - the standardized deployable artifact in BentoML:

bentoml build

Ensure [Docker](https://docs.docker.com/) is running. Generate a Docker container image for deployment:

bentoml containerize summarization:latest

Run the generated image:

docker run --rm -p 3000:3000 summarization:latest

☁️ Deploy on BentoCloud

[BentoCloud](https://www.bentoml.com) provides compute infrastructure for rapid and reliable GenAI adoption. It helps speed up your BentoML development process leveraging cloud compute resources, and simplify how you deploy, scale and operate BentoML in production.

[Sign up for BentoCloud](https://cloud.bentoml.com/signup) for personal access; for enterprise use cases, [contact our team](https://www.bentoml.com/contact).

# After signup, run the following command to create an API token:
bentoml cloud login

# Deploy from current directory:
bentoml deploy

For detailed explanations, read the [Hello World example](https://docs.bentoml.com/en/latest/get-started/hello-world.html).

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 29 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. {
  2. "name": "bentoml",
  3. "description": "Official BentoML agent skills: create Bento projects, containerize them, and deploy them to Kubernetes or EC2",
  4. "owner": {
  5. "name": "BentoML Team",
  6. "url": "https://github.com/bentoml"
  7. },
  8. "plugins": [
  9. {
  10. "name": "bentoml-deploy",
  11. "source": ".",
  12. "strict": false,
  13. "description": "Agent skills for creating BentoML projects, containerizing them, and deploying them to Kubernetes or EC2 with a fully open-source stack",
  14. "author": {
  15. "name": "BentoML Team"
  16. },
  17. "homepage": "https://github.com/bentoml/BentoML",
  18. "license": "Apache-2.0",
  19. "keywords": [
  20. "bentoml",
  21. "bento",
  22. "deployment",
  23. "kubernetes",
  24. "ec2",
  25. "docker"
  26. ]
  27. }
  28. ]
  29. }
README.md · README · 170 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <picture>
  2. <source media="(prefers-color-scheme: dark)" srcset="https://github.com/bentoml/BentoML/assets/489344/d3e6c95d-d224-49a5-9cff-0789f094e127">
  3. <source media="(prefers-color-scheme: light)" srcset="https://github.com/bentoml/BentoML/assets/489344/de4da660-6aeb-4e5a-bf76-b7177435444d">
  4. <img alt="BentoML: Unified Model Serving Framework" src="https://github.com/bentoml/BentoML/assets/489344/de4da660-6aeb-4e5a-bf76-b7177435444d" width="370" style="max-width: 100%;">
  5. </picture>
  6. ## Unified Model Serving Framework
  7. 🍱 Build model inference APIs and multi-model serving systems with any open-source or custom AI models. 👉 [Join our forum](https://forum.modular.com/c/bento/31)!
  8. [![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202-green.svg)](https://github.com/bentoml/BentoML?tab=Apache-2.0-1-ov-file)
  9. [![Releases](https://img.shields.io/github/v/release/bentoml/bentoml.svg)](https://github.com/bentoml/bentoml/releases)
  10. [![CI](https://github.com/bentoml/bentoml/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/bentoml/BentoML/actions/workflows/ci.yml?query=branch%3Amain)
  11. [![Twitter](https://badgen.net/badge/icon/@bentomlai/1DA1F2?icon=twitter&label=Follow)](https://twitter.com/bentomlai)
  12. ## What is BentoML?
  13. BentoML is a Python library for building online serving systems optimized for AI apps and model inference.
  14. - **🍱 Easily build APIs for Any AI/ML Model.** Turn any model inference script into a REST API server with just a few lines of code and standard Python type hints.
  15. - **🐳 Docker Containers made simple.** No more dependency hell! Manage your environments, dependencies and model versions with a simple config file. BentoML automatically generates Docker images, ensures reproducibility, and simplifies how you deploy to different environments.
  16. - **🧭 Maximize CPU/GPU utilization.** Build high performance inference APIs leveraging built-in serving optimization features like dynamic batching, model parallelism, multi-stage pipeline and multi-model inference-graph orchestration.
  17. - **👩‍💻 Fully customizable.** Easily implement your own APIs or task queues, with custom business logic, model inference and multi-model composition. Supports any ML framework, modality, and inference runtime.
  18. - **🚀 Ready for Production.** Develop, run and debug locally. Seamlessly deploy to production with Docker containers or Kubernetes.
  19. ## Getting started
  20. Install BentoML:
  21. ```
  22. # Requires Python≥3.9
  23. pip install -U bentoml
  24. ```
  25. Define APIs in a `service.py` file.
  26. ```python
  27. import bentoml
  28. @bentoml.service(
  29. image=bentoml.images.Image(python_version="3.11").python_packages("torch", "transformers"),
  30. )
  31. class Summarization:
  32. def __init__(self) -> None:
  33. import torch
  34. from transformers import pipeline
  35. device = "cuda" if torch.cuda.is_available() else "cpu"
  36. self.pipeline = pipeline('summarization', device=device)
  37. @bentoml.api(batchable=True)
  38. def summarize(self, texts: list[str]) -> list[str]:
  39. results = self.pipeline(texts)
  40. return [item['summary_text'] for item in results]
  41. ```
  42. ### 💻 Run locally
  43. Install PyTorch and Transformers packages to your Python virtual environment.
  44. ```bash
  45. pip install torch transformers # additional dependencies for local run
  46. ```
  47. Run the service code locally (serving at http://localhost:3000 by default):
  48. ```bash
  49. bentoml serve
  50. ```
  51. You should expect to see the following output.
  52. ```
  53. [INFO] [cli] Starting production HTTP BentoServer from "service:Summarization" listening on http://localhost:3000 (Press CTRL+C to quit)
  54. [INFO] [entry_service:Summarization:1] Service Summarization initialized
  55. ```
  56. Now you can run inference from your browser at http://localhost:3000 or with a Python script:
  57. ```python
  58. import bentoml
  59. with bentoml.SyncHTTPClient('http://localhost:3000') as client:
  60. summarized_text: str = client.summarize([bentoml.__doc__])[0]
  61. print(f"Result: {summarized_text}")
  62. ```
  63. ### 🐳 Deploy using Docker
  64. Run `bentoml build` to package necessary code, models, dependency configs into a Bento - the standardized deployable artifact in BentoML:
  65. ```bash
  66. bentoml build
  67. ```
  68. Ensure [Docker](https://docs.docker.com/) is running. Generate a Docker container image for deployment:
  69. ```bash
  70. bentoml containerize summarization:latest
  71. ```
  72. Run the generated image:
  73. ```bash
  74. docker run --rm -p 3000:3000 summarization:latest
  75. ```
  76. ## Examples
  77. - LLMs: [Llama 3.2](https://github.com/bentoml/BentoVLLM/tree/main/llama3.2-11b-vision-instruct), [Mistral](https://github.com/bentoml/BentoVLLM/tree/main/ministral-8b-instruct-2410), [DeepSeek Distil](https://github.com/bentoml/BentoVLLM/tree/main/deepseek-r1-distill-llama3.1-8b-tool-calling), and more.
  78. - Image Generation: [Stable Diffusion 3 Medium](https://github.com/bentoml/BentoDiffusion/tree/main/sd3-medium), [Stable Video Diffusion](https://github.com/bentoml/BentoDiffusion/tree/main/svd), [Stable Diffusion XL Turbo](https://github.com/bentoml/BentoDiffusion/tree/main/sdxl-turbo), [ControlNet](https://github.com/bentoml/BentoDiffusion/tree/main/controlnet), and [LCM LoRAs](https://github.com/bentoml/BentoDiffusion/tree/main/lcm).
  79. - Embeddings: [SentenceTransformers](https://github.com/bentoml/BentoSentenceTransformers) and [ColPali](https://github.com/bentoml/BentoColPali)
  80. - Audio: [ChatTTS](https://github.com/bentoml/BentoChatTTS), [XTTS](https://github.com/bentoml/BentoXTTS), [WhisperX](https://github.com/bentoml/BentoWhisperX), [Bark](https://github.com/bentoml/BentoBark)
  81. - Computer Vision: [YOLO](https://github.com/bentoml/BentoYolo) and [ResNet](https://github.com/bentoml/BentoResnet)
  82. - Advanced examples: [Function calling](https://github.com/bentoml/BentoFunctionCalling), [LangGraph](https://github.com/bentoml/BentoLangGraph), [CrewAI](https://github.com/bentoml/BentoCrewAI)
  83. Check out the [full list](https://github.com/orgs/bentoml/repositories) for more sample code and usage.
  84. ## Advanced topics
  85. - [Model composition](https://docs.bentoml.com/en/latest/get-started/model-composition.html)
  86. - [Workers and model parallelization](https://docs.bentoml.com/en/latest/build-with-bentoml/parallelize-requests.html)
  87. - [Adaptive batching](https://docs.bentoml.com/en/latest/get-started/adaptive-batching.html)
  88. - [GPU inference](https://docs.bentoml.com/en/latest/build-with-bentoml/gpu-inference.html)
  89. - [Distributed serving systems](https://docs.bentoml.com/en/latest/build-with-bentoml/distributed-services.html)
  90. - [Model loading and Model Store](https://docs.bentoml.com/en/latest/build-with-bentoml/model-loading-and-management.html)
  91. - [Observability](https://docs.bentoml.com/en/latest/build-with-bentoml/observability/index.html)
  92. See [Documentation](https://docs.bentoml.com) for more tutorials and guides.
  93. ## Community
  94. Get involved and join our [Community Forum 💬](https://forum.modular.com/c/bento/31), where thousands of AI/ML engineers help each other, contribute to the project, and talk about building AI products.
  95. To report a bug or suggest a feature request, use
  96. [GitHub Issues](https://github.com/bentoml/BentoML/issues/new/choose).
  97. ### Contributing
  98. There are many ways to contribute to the project:
  99. - Report bugs and "Thumbs up" on [issues](https://github.com/bentoml/BentoML/issues) that are relevant to you.
  100. - Investigate [issues](https://github.com/bentoml/BentoML/issues) and review other developers' [pull requests](https://github.com/bentoml/BentoML/pulls).
  101. - Contribute code or [documentation](https://docs.bentoml.com/en/latest/index.html) to the project by submitting a GitHub pull request.
  102. - Check out the [Contributing Guide](https://github.com/bentoml/BentoML/blob/main/CONTRIBUTING.md) and [Development Guide](https://github.com/bentoml/BentoML/blob/main/DEVELOPMENT.md) to learn more.
  103. - Share your feedback and discuss roadmap plans in our [forum](https://forum.modular.com/c/bento/31).
  104. Thanks to all of our amazing contributors!
  105. <a href="https://github.com/bentoml/BentoML/graphs/contributors">
  106. <img src="https://contrib.rocks/image?repo=bentoml/BentoML" />
  107. </a>
  108. ### Usage tracking and feedback
  109. The BentoML framework collects anonymous usage data that helps our community improve the product. Only BentoML's internal API calls are being reported. This excludes any sensitive information, such as user code, model data, model names, or stack traces. Here's the [code](https://github.com/bentoml/BentoML/blob/main/src/bentoml/_internal/utils/analytics/usage_stats.py) used for usage tracking. You can opt-out of usage tracking by the `--do-not-track` CLI option:
  110. ```bash
  111. bentoml [command] --do-not-track
  112. ```
  113. Or by setting the environment variable:
  114. ```bash
  115. export BENTOML_DO_NOT_TRACK=True
  116. ```
  117. ### License
  118. [Apache License 2.0](https://github.com/bentoml/BentoML/blob/main/LICENSE)

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.