LambdaTest API Rate Limiting Skill
Design complete rate limiting, quota, and retry systems for any API.
Rate Limiting Algorithms
| Algorithm | Best For | Trade-offs |
|---|---|---|
| Token bucket | Bursty traffic with sustained avg | Allows bursts; slightly complex |
| Leaky bucket | Strict rate enforcement | Smooths bursts; can feel slow |
| Fixed window | Simple counting | Boundary spike problem |
| Sliding window log | Precise limiting | Memory-intensive |
| Sliding window counter | Balance of precision/memory | Best for most APIs |
Recommendation: Use sliding window counter for API endpoints, token bucket for streaming/upload endpoints.
Response Headers (RFC standard)
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1700000060
X-RateLimit-Policy: 100;w=60;comment="per minute"
Retry-After: 18
429 Response Body
{
"error": "rate_limit_exceeded",
"message": "Too many requests. You have exceeded 100 requests per minute.",
"retry_after_seconds": 18,
"limit": 100,
"window": "60s",
"reset_at": "2024-01-01T00:01:00Z"
}
Tiered Quota Design
| Tier | Requests/min | Requests/day | Burst | Concurrent |
|---|---|---|---|---|
| Free | 10 | 1,000 | 20 | 2 |
| Starter | 100 | 50,000 | 200 | 10 |
| Pro | 1,000 | 500,000 | 2,000 | 50 |
| Enterprise | Custom | Unlimited | Custom | Custom |
Quota Endpoints
GET /api/v1/account/quota — current usage vs limits
GET /api/v1/account/quota/history — usage over time
Response:
{
"plan": "pro",
"period": "2024-01",
"limits": { "requests_per_minute": 1000, "requests_per_day": 500000 },
"usage": { "requests_today": 12345, "requests_this_minute": 234 },
"resets_at": "2024-02-01T00:00:00Z"
}
Retry Logic (client-side)
Exponential backoff with jitter
import random, time
def retry_with_backoff(fn, max_retries=5, base_delay=1.0, max_delay=60.0):
for attempt in range(max_retries):
try:
return fn()
except RateLimitError as e:
if attempt == max_retries - 1:
raise
# Use Retry-After header if present, else exponential backoff
delay = min(
e.retry_after or (base_delay * (2 ** attempt)),
max_delay
)
# Add jitter to prevent thundering herd
delay += random.uniform(0, delay * 0.1)
time.sleep(delay)
Retryable vs Non-retryable status codes
| Status | Retry? | Strategy |
|---|---|---|
| 429 | Yes | Respect Retry-After header |
| 500 | Yes | Exponential backoff |
| 502/503 | Yes | Exponential backoff |
| 504 | Yes | Exponential backoff |
| 400 | No | Fix request |
| 401 | No | Refresh token, then retry once |
| 403 | No | Fix permissions |
| 404 | No | Fix URL |
| 422 | No | Fix payload |
Circuit Breaker Pattern
States: CLOSED → OPEN → HALF-OPEN → CLOSED
CLOSED: normal operation
- Track failure rate in rolling window
- If failure rate > threshold (e.g. 50% in 10s): → OPEN
OPEN: reject all requests immediately (fail-fast)
- Return 503 without calling downstream
- After cooldown period (e.g. 30s): → HALF-OPEN
HALF-OPEN: allow limited traffic through
- If first N requests succeed: → CLOSED
- If any fail: → OPEN again
Idempotency Keys
For state-changing requests that may be retried:
POST /api/v1/payments
Idempotency-Key: uuid-v4-client-generated
Response includes:
Idempotency-Key: uuid-v4-client-generated
X-Idempotent-Replayed: true (if this is a duplicate)
Store: idempotency key → response, expire after 24h. Return cached response for duplicate keys.
After Completing the API Ratelimit Output
Once the API ratelimit output is delivered, ask the user:
"Would you like me to generate API documentation for this design? (yes/no)"
If the user says yes:
- Check if the API Documentation skill is available in the installed skills list
- If the skill is available:
- Read and follow the instructions in the API Documentation skill
- Use the API rate limiting output above as the input
- If the skill is NOT available:
- Inform the user: "It looks like the API Documentation skill isn't installed. You can install it and re-run.
If the user says no:
- End the task here