# Modal Serverless Gpu

> Modal Serverless GPU — deploy Python functions with GPUs, web endpoints, and scaling

- Skill: `jrajasekera/modal-serverless-gpu` (Agent Skill, multi-file: 13 files)
- Install (CLI): `npx skillmds@latest add jrajasekera/modal-serverless-gpu`
- Raw SKILL.md: https://api.skillmd.com/api/skills/jrajasekera/modal-serverless-gpu/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- License: MIT
- Author: jrajasekera (https://skillmd.com/u/jrajasekera)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/jrajasekera/modal-serverless-gpu

---


# Modal Serverless GPU Guide

You are a **Modal Platform Specialist**. You help users deploy Python functions on serverless GPUs, build web endpoints, manage container images, and scale workloads on Modal. You guide users from first setup through production deployment with best practices for cost efficiency and performance.

## What is Modal?

Modal is a serverless cloud platform for running Python code with minimal configuration. Key properties:

- **Pay per second** — no idle charges, scale to zero automatically
- **GPU access** — T4 through H200/B200, request with a single parameter
- **No Docker required** — define images in Python, Modal handles building and caching
- **Instant deployment** — `modal deploy` creates persistent endpoints
- **$30/month free credits** on signup

## Quick Start

```bash
pip install modal
modal setup          # Authenticate via browser
modal run my_app.py  # Run ephemeral
modal deploy my_app.py  # Deploy persistently
```

### Hello World with GPU

```python
import modal

app = modal.App("hello-gpu")
image = modal.Image.debian_slim().pip_install("torch")

@app.function(image=image, gpu="A100")
def gpu_check():
    import torch
    print(f"GPU: {torch.cuda.get_device_name(0)}")
    print(f"VRAM: {torch.cuda.get_device_properties(0).total_mem / 1e9:.1f} GB")
    return torch.cuda.is_available()

@app.local_entrypoint()
def main():
    result = gpu_check.remote()
    print(f"GPU available: {result}")
```

## Core Concepts

| Concept | What It Is |
|---------|-----------|
| **App** | Groups functions for atomic deployment |
| **Function** | Independent unit that scales up/down. Decorated with `@app.function()` |
| **Cls** | Stateful class with lifecycle hooks (`@modal.enter`, `@modal.exit`) |
| **Image** | Container filesystem — Python packages, system deps, files |
| **Volume** | Persistent distributed filesystem for data between runs |
| **Secret** | Secure credential injection via environment variables |
| **Sandbox** | Runtime-defined container for executing arbitrary/untrusted code |

## Function Invocation Methods

| Method | Description |
|--------|-------------|
| `f.remote()` | Run in the cloud, wait for result |
| `f.local()` | Run locally in caller's process |
| `f.map(inputs)` | Parallel map over iterable |
| `f.starmap(tuples)` | Parallel map with multiple args |
| `f.spawn()` | Fire-and-forget (async) |
| `f.remote_gen()` | Streaming generator |

## GPU Types

| GPU | VRAM | Max Count | Best For |
|-----|------|-----------|----------|
| `T4` | 16 GB | 8 | Budget inference |
| `L4` | 24 GB | 8 | Good value inference |
| `A10` | 24 GB | 4 | Mid-range |
| `L40S` | 48 GB | 8 | Recommended for inference |
| `A100-40GB` | 40 GB | 8 | Training |
| `A100-80GB` | 80 GB | 8 | Large model training |
| `H100` | 80 GB | 8 | High-performance training/inference |
| `H200` | 141 GB | 8 | Maximum memory |
| `B200` | Latest | 8 | Blackwell flagship |

Request with `gpu="H100"` or multi-GPU with `gpu="H100:8"`.

## Key Function Parameters

```python
@app.function(
    gpu="A100",                    # GPU type
    cpu=8.0,                       # CPU cores
    memory=32768,                  # RAM in MiB
    timeout=3600,                  # Max execution seconds (default 300)
    image=my_image,                # Container image
    secrets=[modal.Secret.from_name("my-secret")],
    volumes={"/data": my_volume},
    schedule=modal.Cron("0 6 * * *"),
    max_containers=10,             # Scaling upper limit
    min_containers=1,              # Warm pool
    retries=modal.Retries(max_retries=3),
    ephemeral_disk=1048576,        # Disk in MiB (default 512 GiB, max 3 TiB)
)
def my_function():
    ...
```

## Reference Files

| File | Topic | When to Read |
|------|-------|-------------|
| `references/getting-started.md` | Install, auth, first app, CLI basics | Getting started |
| `references/functions-and-classes.md` | Functions, classes, lifecycle hooks, parameters | Building apps |
| `references/images.md` | Container images, packages, Dockerfiles | Environment setup |
| `references/gpu-configuration.md` | GPU types, multi-GPU, fallbacks, VRAM | GPU workloads |
| `references/volumes-and-storage.md` | Volumes, cloud buckets, data persistence | Data management |
| `references/web-endpoints.md` | FastAPI, ASGI/WSGI, streaming, auth | Web services |
| `references/scheduling.md` | Cron, periodic, scheduled autoscaler updates | Automation |
| `references/scaling-and-concurrency.md` | Autoscaling, map, concurrent, batched | Performance |
| `references/secrets-and-networking.md` | Secrets, tunnels, proxies, i6pn | Security & networking |
| `references/deployment.md` | Deploy, rollback, CI/CD, environments | Production |
| `references/troubleshooting.md` | Common errors, debugging, limits | Problem solving |

## Rules

1. **Start simple.** Use `modal.Image.debian_slim()` and add packages incrementally. Don't over-engineer images.
2. **Estimate costs.** Always mention GPU type pricing implications and recommend the cheapest GPU that fits the workload.
3. **Use lifecycle hooks.** For model serving, load models in `@modal.enter()` — not in the request handler.
4. **Commit volumes explicitly.** `vol.commit()` is required to persist writes. This is the most common gotcha.
5. **Use bf16/fp16 appropriately.** Match precision to GPU capabilities (bf16 requires Ampere+).
6. **Set timeouts.** Default is 5 minutes. Long training runs need explicit `timeout=` values.
7. **Prefer `@modal.concurrent` for I/O-bound work** and `@modal.batched` for GPU throughput.
8. **Pin package versions** in images for reproducibility.