tomevault-io Test Quality Analysis
Expert knowledge for analyzing and improving test quality - detecting test smells, overmocking, insufficient coverage, and other testing anti-patterns.
When to Use This Skill
| Use this skill when... | Use another skill instead when... |
|---|---|
| Reviewing test quality and smells | Writing new unit tests (use vitest-testing) |
| Detecting overmocking or flaky tests | Setting up E2E tests (use playwright-testing) |
| Analyzing test coverage gaps | Validating tests via mutation (use mutation-testing) |
| Improving test maintainability | Generating test data (use property-based-testing) |
| Auditing test suites for anti-patterns | Writing Python tests (use python-testing) |
Core Expertise
Test Quality Dimensions
- Correctness: Tests verify the right behavior
- Reliability: Tests are deterministic and not flaky
- Maintainability: Tests are easy to understand and modify
- Performance: Tests run quickly
- Coverage: Tests cover critical code paths
- Isolation: Tests don't depend on external state
Test Smells - Quick Reference
| Smell | Symptom | Fix |
|---|---|---|
| Overmocking | 3+ mocks per test; mocking pure functions | Mock only I/O boundaries; use real implementations |
| Fragile tests | Break on refactor without behavior change | Test public APIs; use semantic selectors |
| Flaky tests | Non-deterministic pass/fail | Proper async/await; mock time; ensure isolation |
| Test duplication | Copy-pasted setup across tests | Extract to beforeEach(), fixtures, helpers |
| Slow tests | Suite > 10s for unit tests | beforeAll() for expensive setup; parallelize |
| Poor assertions | toBeDefined(), no assertions, mock-only assertions |
Specific matchers; assert outputs and state |
| Insufficient coverage | Critical paths untested | 80%+ on business logic; test error paths and boundaries |
Analysis Tools
TypeScript/JavaScript
vitest --coverage # Coverage report
vitest --coverage --coverage.thresholds.lines=80 # Threshold check
vitest --reporter=verbose # Find slow tests
Python
uv run pytest --cov --cov-report=term-missing # Coverage with missing lines
uv run pytest --cov --cov-fail-under=80 # Threshold check
uv run pytest --durations=10 # Find slow tests
Key Anti-Patterns
Testing Implementation vs Behavior
// BAD: Testing how
test('uses correct algorithm', () => {
const spy = vi.spyOn(Math, 'sqrt')
calculateDistance({ x: 0, y: 0 }, { x: 3, y: 4 })
expect(spy).toHaveBeenCalled()
})
// GOOD: Testing what
test('calculates distance correctly', () => {
const distance = calculateDistance({ x: 0, y: 0 }, { x: 3, y: 4 })
expect(distance).toBe(5)
})
Weak Assertions
// BAD
expect(users).toBeDefined() // Too vague
expect(mockAPI).toHaveBeenCalled() // Testing mock, not behavior
// GOOD
expect(user).toMatchObject({
id: expect.any(Number),
name: 'John',
email: 'john@example.com',
})
Missing Coverage
// BAD: Only tests happy path
test('applies discount', () => {
expect(calculateDiscount(100, 'SAVE20')).toBe(80)
})
// GOOD: Tests all paths
describe('calculateDiscount', () => {
it('applies SAVE20', () => expect(calculateDiscount(100, 'SAVE20')).toBe(80))
it('applies SAVE50', () => expect(calculateDiscount(100, 'SAVE50')).toBe(50))
it('invalid coupon', () => expect(calculateDiscount(100, 'INVALID')).toBe(100))
it('no coupon', () => expect(calculateDiscount(100)).toBe(100))
})
Test Structure (AAA Pattern)
test('user registration flow', async () => {
// Arrange
const userData = { email: 'user@example.com', password: 'secure123' }
const mockEmailService = vi.fn()
// Act
const user = await registerUser(userData, mockEmailService)
// Assert
expect(user).toMatchObject({ email: 'user@example.com', emailVerified: false })
expect(mockEmailService).toHaveBeenCalledWith('user@example.com', expect.any(String))
})
Agentic Optimizations
| Context | Command |
|---|---|
| Coverage check (TS) | vitest --coverage --reporter=dot |
| Coverage check (Python) | uv run pytest --cov --cov-fail-under=80 -q |
| Find slow tests (TS) | vitest --reporter=verbose |
| Find slow tests (Python) | uv run pytest --durations=10 -q |
| Missing lines (Python) | uv run pytest --cov --cov-report=term-missing |
For detailed examples, advanced patterns, and best practices, see REFERENCE.md.
See Also
vitest-testing- TypeScript/JavaScript testingpython-testing- Python pytest testingplaywright-testing- E2E testingmutation-testing- Validate test effectiveness
References
- Test Smells: https://testsmells.org/
- Test Double Patterns: https://martinfowler.com/bliki/TestDouble.html
- Testing Best Practices: https://kentcdodds.com/blog/common-mistakes-with-react-testing-library
Converted and distributed by TomeVault — claim your Tome and manage your conversions.