Testing Strategy

Testing Strategy

Tests are the only correctness instrument you control at build time: shape the suite to the code, not to a decades-old ratio, and make each test assert behaviour a build either passes or fails. Applies to every test under test/ and integration_test/.

Read the reference for the task at hand:

• references/test-layers.md — per-layer harness, imports, teardown, edge tables (pure core, in-memory Drift, headless Notifier, integration).
• references/property-and-fakes.md — fakes-over-mocks, enum-driven fake state, seeded fuzz, independent oracles, absence-of-a-failure-class tests.
• references/coverage-and-budget.md — file-level floors, the coverage-lies-upward fix, the suite-time budget, and the manual-pass handoff.

Run scripts/check_test_hygiene.sh and scripts/run_tests.sh before a PR.

Golden, RTL, and a11y widget mechanics live in widget-golden-and-a11y-testing; this skill governs everything below the pixel.

Non-negotiable rules

1. Shape the suite to the code, not the pyramid. The 70/20/10 numbers trace to a 2011 test-size heuristic (its author said they were "pulled out of a hat") and never described Flutter's unit/widget/integration taxonomy. Test at the cheapest tier that can assert the behaviour: anything expressible as f(input) -> output is a unit or property test, never a pumpWidget. Driving pure logic through the widget tree is slower, flakier, and hides which layer broke.
2. Put business rules in a Flutter-free package and inject a Clock. Domain math lives in pure Dart with zero Flutter/plugin/IO imports, tested with package:test (not flutter_test). The one time type is package:clock's Clock — NEVER DateTime.now(), never a bespoke ClockService. Pure core reads the ambient clock.now(), pinned in tests with withClock(Clock.fixed(t), …); Riverpod/feature code reads the injected clockProvider, overridden with clockProvider.overrideWithValue(Clock.fixed(t)) (seam owned by value-objects-money-and-units). The ban is structural: a pure package declares no Flutter SDK constraint.
3. Assert invariants, not just examples. Every conversion has a round-trip test (decode(encode(x)) == x) and rounding goldens at half-way/boundary values; every universal claim is a seeded fuzz loop (for (var seed = 0; seed < N; seed++)) or property test, checked against an independent oracle — never the production function under test. Print the generated input in reason: so a failure is its own minimal repro. == on integers only; closeTo(x, 1e-9) on doubles.
4. Test the data layer against a real in-memory engine. Use NativeDatabase.memory(), never a mocked DAO — a mocked DAO proves nothing about SQL, constraints, indexes, or migrations. Mock repositories only above the data layer. addTearDown(db.close) and close streams synchronously so reactive .watch() streams do not leak "Timer still pending".
5. Prefer bare-implements fakes over mocks for code you own. A class FakeX implements X (no noSuchMethod superclass) makes an interface change a compile error, models the risk (state, not call-order) as a field, and doubles as the contract's documentation. Reserve mocktail for genuinely external dependencies; never mockito/@GenerateMocks (codegen, no null-safety win).
6. registerFallbackValue for every custom type passed to any()/captureAny() in setUpAll. mocktail throws at runtime, not compile time, on a missing fallback — this is the single most common mocktail failure.
7. Drive Notifiers headlessly with ProviderContainer. Override provider dependencies via overrideWith; never pump a widget to test state. Assert on the exposed AsyncValue/state, drive actions through .notifier, and dispose the container in teardown.
8. Guard one end-to-end acceptance gate. A single realistic scenario asserted to the exact expected result, plus the conservation invariant (parts sum to the whole). It is the test that proves the pieces compose. Back it with a runtime assert tripwire inside the primitive itself (assert(sum(result) == total)) — free in release, catches the bug the moment it happens.
9. Hold an unrecoverable-bug-files floor by diff-review, not a coverage gate. There is no automated percentage gate — coverage is a published report (ci-pipeline-and-gates owns that). Instead hold a 100% floor on the handful of files (migrations, the money/allocate primitive, the parser with wire-format traps) where a gap is silent data loss — enforced by reading the diff, not by counting lines. A directory or global percentage gate rewards vanity tests and exclusion churn. First fix the coverage-lies-upward gap: flutter test --coverage omits files no test imports, so the number overstates safety.
10. Never pumpAndSettle() on an indefinite animation (splash, shimmer, spinner) — it hangs on a 10-minute timeout. Use timed pump(Duration) with fakeAsync.
11. Keep the suite fast, and hand structurally-untestable paths to a manual pass. A suite that costs minutes gets skipped, and a skipped suite is a distrusted one. Anything an emulator cannot reproduce (real audio, OEM device diversity, native surfaces without a Flutter engine) is enumerated in a named manual pre-release pass, not faked green — a green test that proves nothing is worse than an admitted gap.

Tier the test to the code (package:test vs flutter_test)

Pure logic runs under dart test with no widget binding; only UI and plugin-wrapper tests need flutter_test.

// test/domain/money_test.dart — pure Dart, `dart test`, no flutter_test import.
import 'package:test/test.dart';
import 'package:app_core/app_core.dart';

void main() {
  // Round-trip: encoding is lossless across the ISO-4217 exponent range.
  test('minor-units round-trip for exponents 0, 2, 3', () {
    for (final currency in [Currency.jpy, Currency.usd, Currency.kwd]) {
      final money = Money(amountMinor: 12345, currency: currency);
      expect(Money.parse(money.format(), currency), equals(money));
    }
  });
}

Inject the clock; pin it in tests

Production reads clock.now(); tests freeze it so time-dependent logic is deterministic — no wall clock, no flake.

// Production: a Reminder is due when its target time has passed.
bool isDue(Reminder r) => !clock.now().isBefore(r.dueAt);

// test/domain/reminder_test.dart
test('reminder is due exactly at its due instant', () {
  final due = DateTime.utc(2026, 1, 1, 9);
  withClock(Clock.fixed(due), () {
    expect(isDue(Reminder(dueAt: due)), isTrue);
  });
});

For code behind a provider, inject clockProvider and freeze it in a container test with clockProvider.overrideWithValue(Clock.fixed(due)) — same Clock, same fixed instant, driven through Riverpod instead of the ambient zone.

Seeded fuzz against an independent oracle

Twenty lines of package:test give you what a stale property package would, with the generated input as the built-in minimal repro. The oracle must be independent of the function under test.

// test/domain/allocate_test.dart
test('allocate: parts always sum to the input (fuzz)', () {
  final rng = Random(0xC0FFEE);
  for (var seed = 0; seed < 500; seed++) {
    final n = rng.nextInt(20) + 1;
    final total = rng.nextInt(100000);
    final weights = List.generate(n, (_) => rng.nextInt(500));
    final parts = allocate(amountMinor: total, weights: weights);
    expect(parts.fold(0, (a, b) => a + b), equals(total),
        reason: 'seed=$seed total=$total weights=$weights'); // its own repro
  }
});

Real in-memory Drift, never a mocked DAO

// test/data/order_dao_test.dart
void main() {
  late AppDatabase db;
  setUp(() => db = AppDatabase(NativeDatabase.memory()));
  tearDown(() => db.close()); // reactive .watch() streams must not leak timers

  test('CHECK constraint rejects a negative quantity', () {
    expect(
      () => db.orderDao.insertItem(const OrderItem(quantity: -1)),
      throwsA(isA<SqliteException>()), // proves the real constraint, not a stub
    );
  });
}

Headless Notifier tests with ProviderContainer

// test/features/cart/cart_notifier_test.dart
void main() {
  test('addItem publishes the new total', () async {
    final container = ProviderContainer(
      overrides: [orderRepositoryProvider.overrideWith((ref) => _FakeOrderRepo())],
    );
    addTearDown(container.dispose);

    await container.read(cartNotifierProvider.future);
    await container.read(cartNotifierProvider.notifier).addItem(_anItem);

    final state = container.read(cartNotifierProvider);
    expect(state, isA<AsyncData<CartState>>());
    expect(state.requireValue.totalMinor, 1299);
  });
}

ProviderContainer.test() (Riverpod 3.x) auto-disposes and is the preferred form when available; otherwise addTearDown(container.dispose).

Drive timers with fakeAsync

test('debounced search fires once after the window', () {
  fakeAsync((async) {
    final calls = <String>[];
    final debouncer = Debouncer(const Duration(milliseconds: 300));
    debouncer.run(() => calls.add('a'));
    async.elapse(const Duration(milliseconds: 299));
    expect(calls, isEmpty);
    async.elapse(const Duration(milliseconds: 1));
    expect(calls, ['a']);
  });
});

Anti-patterns

• Inverting the pyramid — driving pure f(input) -> output logic through pumpWidget. Slower, flakier, hides the broken layer.
• DateTime.now() / ambient Random() reachable from domain logic. Makes "identical inputs -> identical output" untestable; inject Clock and a seed.
• flutter_test imported into a pure package. Breaks the structural boundary and slows the fastest tier; the package should declare no Flutter SDK constraint.
• A mocked DAO standing in for the database. Proves nothing about SQL, constraints, indexes, or migrations. Use NativeDatabase.memory().
• == on doubles, or covering a universal claim with one lucky example instead of a fuzz loop over an independent oracle.
• A fake that lies about its contract (a dependency that can never fail) — the test then reflects a happy-path stub, not real behaviour. Fakes must expose failure.
• Missing registerFallbackValue for a custom any() argument — a runtime throw no compiler catches.
• pumpAndSettle() on an indefinite indicator — hangs on the 10-minute timeout.
• A global/directory coverage-percentage gate — rewards vanity tests, and the raw number lies upward. Floor the unrecoverable files; publish the rest.
• A green test over a structurally-untestable path (real audio, native tile, device diversity) — worse than an admitted gap, because it stops anyone checking by hand.
• A test with no meaningful expect, shared mutable state across tests, or a vague name like test('works').

Definition of done

• Each test lives at the cheapest tier that can assert it; pure logic under package:test, widget/integration only where the UI/plugin seam is the point.
• Domain logic reads clock.now(), never DateTime.now(); tests pin it with withClock.
• Every conversion has a round-trip test and boundary rounding goldens; every universal claim is a seeded fuzz/property test against an independent oracle, with the input echoed in reason:.
• The data layer is tested against NativeDatabase.memory(), not a mocked DAO; db.close() in teardown.
• Owned collaborators are bare-implements fakes; mocktail is reserved for external deps with registerFallbackValue for every custom any() type.
• Notifiers are tested through ProviderContainer overrides, disposed in teardown; no widget pumped to test state.
• One acceptance gate asserts a realistic scenario exactly and the conservation invariant; the primitive carries a runtime assert tripwire.
• Coverage floors sit on the unrecoverable-bug files; the coverage-lies-upward gap is fixed (untested files included); generated files stripped.
• No pumpAndSettle() on an indefinite animation; timers driven by fakeAsync.
• The suite stays within its time budget; every structurally-untestable path is named in the manual pre-release pass, not faked green.

Related skills

• See widget-golden-and-a11y-testing for golden lanes, the overflow/RTL matrix, and honest a11y limits.
• See value-objects-money-and-units for the allocate() primitive and Clock injection this skill tests.
• See persistence-drift for the DAO/value-object seam the in-memory tests exercise.
• See run-migration for the forward-only migration ritual whose from→to paths get the 100% floor.
• See error-handling-typed-results for the sealed Result/Failure the exhaustive-switch tests assert on.
• See async-safety for the dropped-Future holes the silence-review checklist targets.
• See service-boundary-and-native for the injectable ports the fakes stand in for.
• See ci-pipeline-and-gates for how these lanes and greps are wired as gates.
• See seeded-determinism-and-golden-vectors for pinning a seeded generator with a frozen fingerprint table when output must reproduce across devices and releases.

References

• Flutter testing overview — https://docs.flutter.dev/testing/overview
• Riverpod testing — https://riverpod.dev/docs/essentials/testing
• Drift testing (NativeDatabase.memory) — https://drift.simonbinder.eu/docs/testing/
• package:clock — https://pub.dev/packages/clock
• package:fake_async — https://pub.dev/packages/fake_async
• package:mocktail — https://pub.dev/packages/mocktail