Smoke Test Generator
A structured pattern for API smoke testing with categorised test suites and summary reporting. Adapted from a production test suite that verified 34 endpoints across auth, CRUD, cached audio, story generation, ElevenLabs, and Mistral agent APIs.
Test Categories
| Category |
What It Tests |
Fail = |
| Auth |
Login, token validation, protected routes |
Nothing else works |
| CRUD |
Create, read, update, delete operations |
Data layer broken |
| Cached |
Pre-cached content serves correctly |
Demo will fail |
| Live |
Real API calls complete successfully |
External dependency down |
| Integration |
End-to-end workflows across services |
Pipeline broken |
Pattern
import httpx
import asyncio
BASE_URL = "http://localhost:8000"
results = {"pass": 0, "fail": 0, "skip": 0}
async def test(name: str, category: str, fn):
try:
await fn()
results["pass"] += 1
print(f" ✅ [{category}] {name}")
except Exception as e:
results["fail"] += 1
print(f" ❌ [{category}] {name}: {e}")
async def run_smoke_tests():
async with httpx.AsyncClient(base_url=BASE_URL, timeout=30) as client:
# Auth
await test("Login with valid creds", "auth",
lambda: assert_status(client.post("/login", json={"email": "test@test.com", "password": "test"}), 200))
# CRUD
await test("Create item", "crud",
lambda: assert_status(client.post("/api/items", json={"name": "test"}), 201))
# Cached
await test("Cached content returns 200", "cached",
lambda: assert_status(client.get("/api/cached/1"), 200))
# Integration
await test("Full pipeline completes", "integration",
lambda: assert_status(client.post("/api/pipeline", json={...}), 200))
total = results["pass"] + results["fail"]
print(f"\n{'='*40}")
print(f"Results: {results['pass']}/{total} passed")
if results["fail"] > 0:
print(f"⚠️ {results['fail']} failures — do not demo!")
Files
scripts/smoke_test.py — Example smoke test suite with all categories
1---2name: smoke-test-generator3description: Generate comprehensive API smoke test suites — categorised tests for auth, CRUD, integrations, cached vs live endpoints, with summary reporting. Use when validating API deployments, CI smoke checks, or pre-demo verification. Works with any HTTP API.4---5
6# Smoke Test Generator
7
8A structured pattern for API smoke testing with categorised test suites and summary reporting. Adapted from a production test suite that verified 34 endpoints across auth, CRUD, cached audio, story generation, ElevenLabs, and Mistral agent APIs.
9
10## Test Categories
11
12| Category | What It Tests | Fail = |
13|---|---|---|
14| Auth | Login, token validation, protected routes | Nothing else works |
15| CRUD | Create, read, update, delete operations | Data layer broken |
16| Cached | Pre-cached content serves correctly | Demo will fail |
17| Live | Real API calls complete successfully | External dependency down |
18| Integration | End-to-end workflows across services | Pipeline broken |
19
20## Pattern
21
22```python
23import httpx
24import asyncio
25
26BASE_URL = "http://localhost:8000"
27results = {"pass": 0, "fail": 0, "skip": 0}
28
29async def test(name: str, category: str, fn):
30 try:
31 await fn()
32 results["pass"] += 1
33 print(f" ✅ [{category}] {name}")
34 except Exception as e:
35 results["fail"] += 1
36 print(f" ❌ [{category}] {name}: {e}")
37
38async def run_smoke_tests():
39 async with httpx.AsyncClient(base_url=BASE_URL, timeout=30) as client:
40 # Auth
41 await test("Login with valid creds", "auth",
42 lambda: assert_status(client.post("/login", json={"email": "test@test.com", "password": "test"}), 200))
43
44 # CRUD
45 await test("Create item", "crud",
46 lambda: assert_status(client.post("/api/items", json={"name": "test"}), 201))
47
48 # Cached
49 await test("Cached content returns 200", "cached",
50 lambda: assert_status(client.get("/api/cached/1"), 200))
51
52 # Integration
53 await test("Full pipeline completes", "integration",
54 lambda: assert_status(client.post("/api/pipeline", json={...}), 200))
55
56 total = results["pass"] + results["fail"]
57 print(f"\n{'='*40}")
58 print(f"Results: {results['pass']}/{total} passed")
59 if results["fail"] > 0:
60 print(f"⚠️ {results['fail']} failures — do not demo!")
61```
62
63## Files
64
65- `scripts/smoke_test.py` — Example smoke test suite with all categories
modbender