Nebo Testing QA

TESTING-QA SuperSkill

Заменяет: 8 скилов кластера TESTING-QA (job-analytics, skyvern, fastmonai, opensquad, specflow, tdd-workflows, tdd-mastery, testing-strategies) Триггеры: TDD, тесты, pytest, coverage, code review QA, BDD, mutation testing Версия: 1.0 | 26.03.2026

КОГДА ПРИМЕНЯТЬ

• Написание тестов / TDD цикл
• Git diff QA validation (PR review)
• Architecture plan review (staff review)
• Coverage analysis и пороги
• Property-based / mutation / snapshot / contract testing
• BDD / SpecFlow
• AI-powered test generation

АТОМЫ (уникальные инструкции)

TDD Red-Green-Refactor

1. RED: Напиши failing test, определяющий desired behavior — ПЕРЕД любым production code
2. GREEN: Напиши МИНИМАЛЬНЫЙ код чтобы тест прошёл — не больше
3. REFACTOR: Очисти код, все тесты зелёные

• Цикл = 2-10 минут; если дольше — scope слишком большой
• Outside-in TDD для features/user-stories; Inside-out TDD для компонентов/библиотек
• Chicago School (state-based) vs London School (mockist/interaction-based)

Test Structure

• Arrange-Act-Assert в каждом тесте
• Naming: test_<unit>_<scenario>_<expected> (Python/Go) или it("should <behavior> when <condition>") (JS/TS)
• Каждый тест independent и self-contained — нет shared mutable state
• Test behavior, NOT implementation — тесты должны пережить рефакторинг

Coverage Rules

• 80% line coverage minimum в CI
• 75% branch coverage minimum
• Exclude: generated code, type definitions, config files
• Конкретные команды:
  • vitest run --coverage --coverage.thresholds.lines=80 --coverage.thresholds.branches=75
  • pytest --cov=src --cov-fail-under=80 --cov-branch
  • go test -coverprofile=cover.out -coverpkg=./... ./...
• НИКОГДА тесты только ради coverage numbers — test behavior

Test Pyramid

• 70% unit / 20% integration / 10% E2E
• Unit: одна функция/класс, <100ms, mock все external dependencies
• Integration: module boundaries, <5s, реальная DB/FS допустима
• E2E: full user flow, <30s, full stack
• Mark slow: @pytest.mark.slow, @pytest.mark.e2e

Mocking Guidelines

• Mock на границах: HTTP clients, databases, file systems, clocks
• НИКОГДА mock unit under test
• Prefer fakes (in-memory implementations) над mocks для repositories
• Assert on behavior, не на mock call counts
• DI > module mocking

Git Diff QA Validation (skyvern)

• QA diff-driven: git diff → understand affected behavior → validate
• Classify: frontend/browser, backend API, backend-internal, mixed
• Mixed: backend FIRST, потом frontend (broken backend → untrustworthy frontend)
• Backend-internal: compile/type/lint + targeted tests — НЕ browse UI
• Frontend: browser automation с deterministic DOM assertions (document.querySelector)
• Backend API: changed endpoints → happy path + empty/not-found + invalid input + mutation follow-up
• Health gate: error messages, blank pages, auth redirects, JS errors
• Network check: performance.getEntriesByType('resource').filter(e => e.responseStatus >= 400)
• Post QA report как sticky PR comment с <!-- skyvern-qa-report --> marker

Contract Testing (Pact)

• .given(state).uponReceiving(description).withRequest().willRespondWith().executeTest()
• Consumer expectations match provider capabilities без обоих запущенных сервисов
• Contract violations = build fails — без исключений

SpecFlow BDD Contracts

• YAML в docs/contracts/: contract_meta (id, version, covers_reqs) + rules (non_negotiable: forbidden_patterns + required_patterns) + compliance_checklist
• REQ ID format: [DOMAIN]-[NUMBER] (ARCH-001, AUTH-001, FEAT-001, SEC-001)
• Architecture contracts ПЕРЕД feature contracts
• Contract tests ПЕРЕД build; journey tests ПОСЛЕ build
• Override: user must explicitly say override_contract: [CONTRACT_ID]

Journey Testing (BDD E2E)

• Journey = Definition of Done: feature complete когда journeys pass
• test.describe('Journey: [J-REQ-ID]') с Playwright step-by-step
• Plain English requirements → REQ IDs → contract YAML → contract tests → journey tests

Property-Based Testing

• fast-check (JS/TS), Hypothesis (Python), QuickCheck (Haskell)
• Assert invariants: output length preservation, sort order, idempotency
• fc.assert(fc.property(arbitraries, predicate))
• Для algorithmic code specifically

Mutation Testing

• Validate test suite quality: kill mutants = good tests
• If mutants survive → tests insufficient
• Integrate в CI pipeline

Snapshot Testing

• toMatchSnapshot() для component render output
• Inline snapshots (toMatchInlineSnapshot()) для small outputs
• Review snapshot diffs carefully в code review — НИКОГДА blindly update

Staff Review (Architecture Plans)

• Check: failure modes, race conditions, data integrity, rollback, security, performance cliffs
• YAGNI: interfaces с single implementation = violation
• Completeness: success criteria, testing strategy, observability, docs, deployment
• Severity: Critical (blocks approval), Medium (should address), Minor (nice to have)
• Verdict: GO, GO WITH CONDITIONS, REVISE
• Prefer reversible decisions when uncertain

Integration Testing с Containers

• Testcontainers (@testcontainers/postgresql) для real DB
• beforeAll: start container + migrate; afterAll: close + stop
• Timeout: 60s для container startup

ЗАПРЕЩЕНО

• Test implementation details вместо behavior
• Тесты которые проходят при удалении кода (tautological)
• Shared mutable state между тестами
• Игнорировать flaky tests (fix root cause!)
• Test private methods directly
• Giant test setup скрывающий intent
• Tests depending on execution order
• Mock всё в integration tests (use real dependencies)
• Test trivial getters/setters при пропуске edge cases

БЫСТРЫЙ СТАРТ

1. TDD: RED (failing test) → GREEN (minimal code) → REFACTOR; цикл 2-10 мин
2. Coverage: 80% lines / 75% branches; pyramid 70/20/10; mock только на границах
3. Diff QA: classify (frontend/backend/mixed) → backend first → deterministic assertions → sticky PR comment