# Hf Perftest

> hf-perftest — Gradio Performance Benchmarking

- Skill: `gradio-app/hf-perftest` (Agent Skill, multi-file: 18 files)
- Install (CLI): `npx skillmds@latest add gradio-app/hf-perftest`
- Raw SKILL.md: https://api.skillmd.com/api/skills/gradio-app/hf-perftest/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: gradio-app (https://skillmd.com/u/gradio-app)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/gradio-app/hf-perftest

---


# hf-perftest — Gradio Performance Benchmarking

## Installation

```bash
pip install hf-perftest
```

## Commands

### List built-in apps

```bash
hf-perftest list-apps
```

Built-in apps can be used by name (no file path needed): `echo_text`, `file_heavy`, `image_to_image`, `streaming_chat`, `stateful_counter`, `llm_chat`, `text_to_image`, `audio_to_audio`, `video_to_video`.

### Local benchmark

```bash
hf-perftest run \
    --app echo_text \
    --tiers 1,10,100 \
    --requests-per-user 10 \
    --output-dir results
```

Key options:
- `--app` — Built-in app name or path to a Gradio app file (required)
- `--tiers` — Comma-separated concurrent user counts (default: 1,10,100)
- `--requests-per-user` — Rounds per tier (default: 10)
- `--mode burst|wave` — Simultaneous or staggered requests (default: burst)
- `--concurrency-limit` — App concurrency limit (default: 1, "none" for unlimited)
- `--mixed-traffic` — Add background page loads, uploads, and downloads alongside predictions
- `--num-workers` — Number of Gradio workers via GRADIO_NUM_WORKERS (default: 1)
- `--port` — App port (default: 7860, auto-increments if occupied)
- `--api-name` — Target API endpoint (auto-detected if omitted)

### Remote benchmark on HF Jobs

Single branch:
```bash
hf-perftest run-remote run \
    --apps echo_text streaming_chat \
    --branch main \
    --hardware cpu-upgrade \
    --tiers 1,10,100
```

A/B test:
```bash
hf-perftest run-remote ab \
    --apps echo_text file_heavy \
    --base main \
    --branch my-optimization \
    --hardware cpu-upgrade \
    --tiers 1,10,100
```

Profile a HF Space:
```bash
hf-perftest run-remote run \
    --apps owner/space-name \
    --sidecar prompts.json \
    --api-name /generate \
    --branch main \
    --hardware gpu-l4-1
```

Additional remote options:
- `--hardware` — HF Jobs flavor (default: cpu-basic)
- `--sidecar` — Prompt files for spaces
- `--timeout` — Job timeout (default: 90m)
- `--dry-run` — Preview without submitting
- `--run-name` — Label for the run

### Result schema

```bash
hf-perftest result-schema
```

Prints the directory structure of benchmark results.

## Sidecar Prompt Files

For apps with non-text inputs, create a `.prompts.json` sidecar file. Two formats:

**String list** (text-only inputs — replaces the first text input):
```json
["A cat sitting on a windowsill", "Sunset over a mountain lake"]
```

**List of lists** (full data payloads — sent as-is):
```json
[
    ["A cat sitting on a windowsill", 1024, 1024, 4, 42, true],
    ["Sunset over a mountain lake", 1024, 1024, 4, 42, true]
]
```

## Interpreting Results

Results are saved to `<output-dir>/<timestamp>/summary.json` with per-tier breakdowns:
- `client_summary` — p50/p90/p95/p99 client latency in ms, success rate
- `server_summary` — Per-phase server timing (queue_wait, preprocess, fn_call, postprocess, total)
- `background_traffic` — (if --mixed-traffic) p50/p90/p99 for page loads, uploads, downloads

Always validate with multiple runs — single runs may be affected by system variance.

## Monitoring Remote Jobs

```bash
hf jobs logs <job_id>
hf jobs inspect <job_id>
```