Bentoml Create Bento

Create a BentoML project

A Bento is the artifact bentoml build produces; a project is the directory it is built from, normally one service.py plus its dependencies. @bentoml.service on a class makes it a Service — the unit that gets deployed and scaled, and there may be several in one bento — and @bentoml.api on a method makes it an HTTP endpoint whose schema is the method's type annotations. BentoML supplies the server, the OpenAPI spec, the Swagger UI, the client, health checks and metrics. You write the class.

Two paths, same steps 2-5: A from scratch (§1A) · B convert existing code (§1B).

It also covers an existing project: adding an endpoint, switching an API to streaming or adaptive batching, moving long work to a background task, splitting one Service into several, and diagnosing a bento that will not start, rejects valid input, or serves one request at a time (last section).

What you produce

No Dockerfile, no uvicorn/gunicorn call, no main(), no route table, no request/response plumbing. If you are writing any of those, you are working against the framework.

Step 0 — get the anchor case

Before writing anything, get one concrete input and its expected output from the user (or from the existing code's tests/README). This is the anchor: §4 verifies against it. Without it you can only confirm that a bento returns something, which is how a wrong one ships.

If the user cannot supply one, derive it from the model card / existing code and state the assumption explicitly.

Step 1A — new project: ask once

Ask these together, with defaults, in one round. Accept "you decide" for any of them.

Do not ask about workers, threads, replicas, ports, batching, registries or timeouts. The defaults are correct until something is measured, and every one of them is a one-line change later.

Then scaffold the file set above and go to §2. Keep the first version to one Service and one API even if the target is bigger — get it green, then grow.

Step 1B — convert existing code

Read the existing code first. Find and write down: (1) the inference entry point, (2) where the weights live, (3) the real dependency list, (4) the anchor case. Then translate:

Rules for a conversion: do not rewrite the model code. Import it, call it, keep it importable so the old and new paths can be diffed. Convert the smallest slice that serves the anchor case, verify parity (§4), then move the rest. Detail and per-source playbooks: conversion.

Step 2 — write service.py

from __future__ import annotations

import bentoml
from bentoml.models import HuggingFaceModel

with bentoml.importing():           # heavy/optional imports: skipped at build, required at serve
    from transformers import pipeline


@bentoml.service(
    resources={"cpu": "2"},         # deployment hint (see rules), not a local limit
    traffic={"timeout": 30},        # seconds; default 60
)
class Summarization:
    model_path = HuggingFaceModel("sshleifer/distilbart-cnn-12-6")   # class scope, always

    def __init__(self) -> None:     # once per worker
        self.pipeline = pipeline("summarization", model=self.model_path)

    @bentoml.api                    # -> POST /summarize
    def summarize(self, text: str) -> str:
        return self.pipeline(text)[0]["summary_text"]

Rules that are not guessable from the API surface:

Multiple Services in one service.py: give each its own @bentoml.service, wire them with dep = bentoml.depends(Other) at class scope, and call self.dep.method(...) like a local method. The Service you serve is the entry service; BentoML starts its dependencies with it. Split only for a real reason — different hardware, independent scaling, or a shared downstream — since each split adds a network hop. Details: distributed services.

IO types beyond str/int/dict (numpy, pandas, PIL.Image, Path, pydantic models, root input, validators, streaming, batching): io-types. Config keys: service-config. Task queues, LLM streaming, model composition, FastAPI mounts, Gradio UIs: recipes.

Step 3 — declare the runtime environment

image = bentoml.images.Image(python_version="3.11").requirements_file("requirements.txt")

@bentoml.service(image=image)
class Summarization: ...

• Image is the current API (BentoML >= 1.3.20) and lives next to the code it describes. Chainable: .python_packages(...), .requirements_file(...), .system_packages(...), .run("cmd"), .run_script("scripts/setup.sh"), .build_include(...). .run() is position-sensitive: before .python_packages() runs before pip, after it runs after.
• bentoml itself is added automatically. Don't list it.
• Versions are locked at build time unless you pass lock_python_packages=False.
• A multi-Service bento uses only the entry Service's image; per-Service images are not supported yet, so put every dependency there.
• .bentoignore (gitignore syntax) is what keeps the bento small — always exclude the venv, __pycache__/, datasets and checkpoints.

Step 4 — serve and verify (never skip)

bentoml serve                      # resolves bentofile.yaml -> pyproject.toml -> service.py
bentoml serve service:Summarization --reload -p 3000    # explicit target, dev reload

Then, in this order — each check catches a different class of mistake:

curl -sf localhost:3000/readyz                       # 1. it started and loaded the model
curl -s localhost:3000/docs.json | head -c 400       # 2. the schema is what you intended
curl -s -X POST localhost:3000/summarize \           # 3. the anchor case, judged by its BODY
     -H 'Content-Type: application/json' -d '{"text": "..."}'

Judge the response body against the anchor from §0, not the status code — a bento that returns null, an empty string or a stack trace with a 200 is a failed verification. For a conversion, run the original code on the same input and diff the two outputs; report the comparison, and if they differ, say so rather than adjusting the expectation.

Commit the anchor as a test. No server needed:

import pytest
from starlette.testclient import TestClient
from service import Summarization

@pytest.fixture(scope="session")            # to_asgi() ONCE per process: it registers
def client():                               # Prometheus collectors, and a second call
    with TestClient(Summarization.to_asgi()) as c:   # raises "Duplicated timeseries"
        yield c

def test_summarize(client):                 # TestClient must be a context manager
    r = client.post("/summarize", json={"text": "..."})
    assert r.status_code == 200 and "expected substring" in r.text

Summarization() can also be instantiated directly and called as plain Python — the fastest loop while iterating. More: testing.

Step 5 — build and hand off

bentoml build          # -> summarization:xxxxxxxx  (packages the cwd, minus .bentoignore)
bentoml list
bentoml serve summarization:latest      # serve the built artifact, not the source

The bento name is the entry Service's name in snake_case (DiabetesRisk -> diabetes_risk) — that, not the class name, is what the deploy skills and bentoml containerize take.

A built Bento is the input to bentoml-containerize (image + registry push), then bentoml-k8s-deploy or bentoml-ec2-deploy. Tell the user that is the next step; do not build images or touch a cluster from this skill.

When it goes wrong

References

Local, load when the task needs them:

Upstream docs (authoritative, versioned): Services, IO types, runtime environment, model loading, distributed services, batching, tasks, streaming, ASGI, hooks, clients, testing, build options, SDK reference, examples.