# Test Coverage Categories
> Comprehensive systematic approach to achieving complete test coverage through structured category-based testing
- Skill: `snoodleboot-io/test-coverage-categories` (Agent Skill)
- Install (CLI): `npx skillmds@latest add snoodleboot-io/test-coverage-categories`
- Raw SKILL.md: https://api.skillmd.com/api/skills/snoodleboot-io/test-coverage-categories/raw
- Safety review: CAUTION (external: skill-scanner FAIL, skillspector CAUTION)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: snoodleboot-io (https://skillmd.com/u/snoodleboot-io)
- Updated: 2026-08-19
- Page: https://skillmd.com/skills/snoodleboot-io/test-coverage-categories
---
## Coverage Categories
A systematic approach to comprehensive test coverage. Work through these categories in order to ensure your tests catch all failure modes, edge cases, and security issues.
---
### Category 1: HAPPY PATH
**Definition:** Expected inputs produce expected outputs with no errors.
**Purpose:** Verify core functionality works under normal conditions.
**What to test:**
- Valid, typical inputs
- Expected return values
- Expected side effects (DB writes, API calls)
- Expected state changes
**Example:**
```python
def test_create_user_happy_path():
"""Test creating user with valid data succeeds."""
user_data = {
"email": "alice@example.com",
"name": "Alice",
"password": "SecurePass123!",
}
user = create_user(user_data)
# Assert expected return value
assert user.email == "alice@example.com"
assert user.name == "Alice"
assert user.id is not None
# Assert expected side effects
assert db.users.count() == 1
assert user.created_at is not None
```
**Common mistakes:**
- ❌ Only testing happy path (ignoring edge cases)
- ❌ Not verifying side effects (DB writes, logs, events)
- ❌ Using unrealistic inputs (test with production-like data)
---
### Category 2: BOUNDARY VALUES
**Definition:** Test at the edges of valid ranges (min, max, exactly at limit, one over limit).
**Purpose:** Catch off-by-one errors, overflow issues, and boundary-related bugs.
**What to test:**
- Minimum valid value
- Maximum valid value
- Exactly at limit (e.g., string length exactly 255 if that's the limit)
- One below minimum (should fail)
- One above maximum (should fail)
- Empty collections (length 0)
- Single-item collections (length 1)
**Example:**
```python
def test_username_length_boundaries():
"""Test username validation at boundary values."""
# Minimum length (3 chars) - valid
assert is_valid_username("abc") is True
# Below minimum (2 chars) - invalid
assert is_valid_username("ab") is False
# Maximum length (20 chars) - valid
assert is_valid_username("a" * 20) is True
# Above maximum (21 chars) - invalid
assert is_valid_username("a" * 21) is False
# Empty string - invalid
assert is_valid_username("") is False
def test_pagination_boundaries():
"""Test pagination at boundary values."""
# Page size = 1 (minimum)
results = get_users(page=1, size=1)
assert len(results) == 1
# Page size = 100 (maximum)
results = get_users(page=1, size=100)
assert len(results) <= 100
# Page size = 0 (invalid)
with pytest.raises(ValidationError):
get_users(page=1, size=0)
# Page size = 101 (over max)
with pytest.raises(ValidationError):
get_users(page=1, size=101)
```
**Common boundary conditions:**
- String length (min, max)
- Numeric ranges (min, max, 0, negative, positive)
- Collection size (0, 1, max)
- Date ranges (start, end, exactly at cutoff)
- Pagination (first page, last page, page beyond last)
---
### Category 3: EMPTY / NULL / ZERO
**Definition:** Test with absent, null, zero, or empty values for each nullable input.
**Purpose:** Catch null pointer exceptions, division by zero, and missing data handling bugs.
**What to test:**
- `None` / `null` / `undefined` for nullable fields
- Empty strings (`""`)
- Empty collections (`[]`, `{}`)
- Zero values (`0`, `0.0`)
- False boolean values
**Example:**
```python
def test_calculate_average_with_empty_list():
"""Test average calculation with empty list."""
# Empty list should raise or return None (depending on design)
with pytest.raises(ValueError, match="Cannot calculate average of empty list"):
calculate_average([])
def test_get_user_by_email_with_none():
"""Test user lookup with None email."""
with pytest.raises(ValidationError, match="Email cannot be None"):
get_user_by_email(None)
def test_format_address_with_missing_fields():
"""Test address formatting with missing optional fields."""
address = {
"street": "123 Main St",
"city": "Springfield",
"state": None, # Optional field
"zip": "", # Empty string
}
# Should handle missing fields gracefully
formatted = format_address(address)
assert "123 Main St" in formatted
assert "Springfield" in formatted
# Should not include state or zip if missing
def test_divide_by_zero():
"""Test division with zero divisor."""
with pytest.raises(ZeroDivisionError):
divide(10, 0)
```
**Checklist for nullable inputs:**
- [ ] Test each nullable parameter with `None`
- [ ] Test each string parameter with `""`
- [ ] Test each list parameter with `[]`
- [ ] Test each dict parameter with `{}`
- [ ] Test each numeric parameter with `0`
---
### Category 4: ERROR CASES
**Definition:** Test when dependencies fail, resources are unavailable, or external calls raise exceptions.
**Purpose:** Verify error handling, logging, and graceful degradation.
**What to test:**
- Database connection failures
- Network timeouts
- External API errors (4xx, 5xx)
- File not found
- Permission denied
- Resource exhaustion (out of memory, disk full)
- Invalid state transitions
**Example:**
```python
def test_save_user_when_database_unavailable(mocker):
"""Test user save handles DB connection failure gracefully."""
# Mock database to raise connection error
mocker.patch.object(db, "save", side_effect=ConnectionError("DB unavailable"))
user = User(email="alice@example.com")
# Should raise specific error (not generic exception)
with pytest.raises(DatabaseError, match="Failed to save user"):
save_user(user)
# Should log error
assert "DB unavailable" in captured_logs
def test_fetch_user_when_api_returns_500(mocker):
"""Test user fetch handles API 500 error."""
mocker.patch("requests.get", return_value=MockResponse(status=500))
with pytest.raises(ExternalAPIError):
fetch_user_from_external_api("user123")
def test_process_payment_when_gateway_timeout(mocker):
"""Test payment processing handles gateway timeout."""
mocker.patch.object(
payment_gateway,
"charge",
side_effect=TimeoutError("Gateway timeout"),
)
result = process_payment(amount=100, card_token="tok_123")
# Should return failure result (not raise)
assert result.status == "failed"
assert "timeout" in result.error.lower()
# Should not charge user
assert payment_gateway.charge.call_count == 1 # No retries on timeout
```
**Common error scenarios:**
- Network failures (timeout, connection refused, DNS error)
- Authentication failures (invalid token, expired token)
- Authorization failures (insufficient permissions)
- Validation failures (invalid input format)
- Resource not found (404)
- Conflict (409 - duplicate, race condition)
- Rate limiting (429)
- Server errors (500, 502, 503)
---
### Category 5: CONCURRENT / ORDERING
**Definition:** If function has state or depends on execution order, test with different orderings or concurrent access.
**Purpose:** Catch race conditions, deadlocks, and state corruption from concurrent access.
**What to test:**
- Concurrent reads
- Concurrent writes
- Read-modify-write races
- Order of operations (does A→B produce same result as B→A?)
- State transitions (valid vs invalid sequences)
**Example:**
```python
import asyncio
import pytest
@pytest.mark.asyncio
async def test_concurrent_counter_increments():
"""Test counter handles concurrent increments correctly."""
counter = Counter()
# Run 100 concurrent increments
tasks = [counter.increment() for _ in range(100)]
await asyncio.gather(*tasks)
# Should be exactly 100 (no lost updates)
assert counter.value == 100
def test_order_state_transitions():
"""Test order state transitions enforce valid sequences."""
order = Order()
# Valid sequence: pending → confirmed → shipped → delivered
order.confirm()
assert order.status == OrderStatus.CONFIRMED
order.ship()
assert order.status == OrderStatus.SHIPPED
order.deliver()
assert order.status == OrderStatus.DELIVERED
def test_order_invalid_state_transition():
"""Test order prevents invalid state transitions."""
order = Order()
# Invalid: skip from pending directly to delivered
with pytest.raises(InvalidStateTransitionError):
order.deliver() # Must confirm and ship first
@pytest.mark.asyncio
async def test_concurrent_stock_updates():
"""Test concurrent stock updates don't oversell."""
product = Product(stock=10)
# 20 concurrent purchase attempts (should only succeed 10 times)
tasks = [product.purchase(quantity=1) for _ in range(20)]
results = await asyncio.gather(*tasks, return_exceptions=True)
# Exactly 10 should succeed
successes = [r for r in results if not isinstance(r, Exception)]
assert len(successes) == 10
# Stock should be 0
assert product.stock == 0
```
**When to test concurrency:**
- ✅ Shared mutable state (counters, caches, inventories)
- ✅ Database writes (race conditions, lost updates)
- ✅ File writes (concurrent modifications)
- ✅ Resource pools (connection pools, thread pools)
**When to skip:**
- ❌ Pure functions (no side effects)
- ❌ Read-only operations
- ❌ Single-threaded contexts
---
### Category 6: AUTHORIZATION BOUNDARIES
**Definition:** Test that functions enforce access control (who can call it, what they can access).
**Purpose:** Catch security vulnerabilities where unauthorized users can access restricted resources.
**What to test:**
- Unauthenticated access (no token)
- Authenticated but unauthorized (wrong role/permissions)
- Accessing resources owned by other users
- Privilege escalation attempts
**Example:**
```python
def test_get_user_requires_authentication():
"""Test user retrieval requires authentication."""
# No auth token provided
with pytest.raises(Unauthenticated):
get_user(user_id="123", auth_token=None)
def test_update_user_requires_ownership():
"""Test user update requires ownership or admin role."""
alice = create_user(email="alice@example.com")
bob = create_user(email="bob@example.com")
# Bob tries to update Alice's profile (should fail)
with pytest.raises(Forbidden):
update_user(
user_id=alice.id,
data={"name": "Hacked"},
auth_user_id=bob.id,
)
def test_delete_order_requires_admin_role():
"""Test order deletion requires admin role."""
user = create_user(email="user@example.com", role="user")
admin = create_user(email="admin@example.com", role="admin")
order = create_order(user_id=user.id)
# Regular user cannot delete order
with pytest.raises(Forbidden):
delete_order(order_id=order.id, auth_user_id=user.id)
# Admin can delete order
delete_order(order_id=order.id, auth_user_id=admin.id)
assert get_order(order.id) is None
def test_list_users_filters_by_organization():
"""Test user list enforces organization boundary."""
org_a = create_organization(name="Org A")
org_b = create_organization(name="Org B")
alice = create_user(email="alice@a.com", org_id=org_a.id)
bob = create_user(email="bob@b.com", org_id=org_b.id)
# Alice should only see users in Org A
users = list_users(auth_user_id=alice.id)
assert len(users) == 1
assert users[0].id == alice.id
```
**Authorization patterns to test:**
- Resource ownership (user can only access their own data)
- Role-based access (admin vs user vs guest)
- Permission-based access (specific permissions required)
- Organization/tenant boundaries (multi-tenant isolation)
- Hierarchical access (manager can access subordinates' data)
---
### Category 7: ADVERSARIAL INPUTS
**Definition:** Test with malicious or malformed inputs designed to exploit vulnerabilities.
**Purpose:** Catch security vulnerabilities (SQL injection, XSS, path traversal, etc.) and robustness issues.
**What to test:**
- **SQL injection:** SQL fragments in string inputs
- **XSS:** Script tags, JavaScript in text fields
- **Path traversal:** `../../../etc/passwd` in file paths
- **Command injection:** Shell metacharacters in system calls
- **Unicode/emoji:** Non-ASCII characters
- **Control characters:** Null bytes (`\0`), newlines, tabs
- **Extremely long strings:** 1MB strings, 10K-character names
- **Format string attacks:** `%s`, `%n` in format strings
- **LDAP injection:** LDAP query fragments
- **XML injection:** XML entities, CDATA
**Example:**
```python
def test_search_users_with_sql_injection_attempt():
"""Test user search is protected against SQL injection."""
# Attempt SQL injection
malicious_query = "'; DROP TABLE users; --"
# Should NOT execute SQL, should treat as literal string
results = search_users(query=malicious_query)
# Should return empty results (no match) or error, NOT drop table
assert isinstance(results, list)
# Verify table still exists
assert db.table_exists("users")
def test_create_user_with_xss_in_name():
"""Test user creation sanitizes XSS in name field."""
malicious_name = ""
user = create_user(email="test@example.com", name=malicious_name)
# Should escape or strip script tags
assert "