Mobile QA Process

Mobile QA Process

Instructions

QA for mobile is about catching what automation cannot: visual regressions across devices, real-world network conditions, store compliance, and UX friction. A working QA process is lightweight, predictable, and repeated every release.

1. QA Cadence

Tie QA to the release train, not to a calendar.

• Pre-merge: automated checks + code review (see mobile-testing-strategy).
• Pre-release candidate: smoke test on internal testing track.
• Release candidate: full regression + exploratory bug bash.
• Post-release: production smoke + monitored rollout (see staged-rollouts).

2. The Release Checklist

Every release runs the same checklist. Store it as a template in the repo.

# Release Checklist v<date>

## Versioning
- [ ] versionName matches changelog
- [ ] versionCode / build number strictly increases
- [ ] Git tag applied on the release commit

## Build
- [ ] Release build succeeds on CI with the production flavor
- [ ] App size delta < 10% vs last release (investigate otherwise)
- [ ] Startup time under 2s on reference devices

## Functionality
- [ ] Smoke pass on iOS and Android (latest + oldest supported)
- [ ] Regression suite green
- [ ] Critical deep links verified on both platforms
- [ ] Push notifications verified (alert + data) end to end

## Compliance
- [ ] Privacy nutrition labels / Data Safety form match SDK inventory
- [ ] Permission prompts have correct Info.plist strings / manifest rationales
- [ ] Export compliance answered (iOS encryption declaration)
- [ ] Accessibility spot-check (VoiceOver, TalkBack, Dynamic Type)

## Store
- [ ] Screenshots current for all required locales and device sizes
- [ ] What's New notes drafted
- [ ] Age rating reviewed if content changed
- [ ] Staged rollout plan written (start %, halt criteria)

## Observability
- [ ] Crash-free sessions baseline recorded from previous version
- [ ] Analytics dashboard links noted
- [ ] Kill switches and feature flags verified

3. Regression Suite

A living, manual document for what automated tests do not cover:

• Sign-in with each auth provider (email, SSO, passkey) including failure paths.
• Offline flows on each platform.
• Push notification variants (cold start, foreground, backgrounded, actions).
• Payment sandbox purchases + restore.
• Camera, microphone, photo picker, notifications permissions in Settings-off state.
• In-app update flow (Android) and forced-update gate (both).
• Deep links from web, email, and notifications.
• Backgrounding during long operations (upload, download).

Keep the suite under 60 minutes for a single platform pass.

4. Exploratory Bug Bash

• Invite engineering, design, PM, support, and at least two outsiders.
• 60-90 minutes, two platforms.
• Each participant gets a persona and a focus area (onboarding, payments, settings, edge cases).
• Participants file bugs with reproduction steps, device, and OS version. No discussion during the session.
• Triage immediately afterwards.

Bug bashes surface issues automation never will: unreadable contrast, copy mismatches, subtle animation jank.

5. Real-World Conditions

Spend part of QA outside the office/Wi-Fi:

• Elevator, subway, tunnel -> reconnect behavior.
• Low-power mode / battery saver -> background work disabled.
• Dark mode, dynamic type at 200%.
• Right-to-left language (pseudo-locale on iOS, RTL pseudolocale on Android).
• Network shaping: Network Link Conditioner on iOS, adb shell tc / charles / proxyman for throttling.

6. Defect Tracking Hygiene

• Every bug has: platform, OS, device, build, reproduction steps, severity, screenshot/video.
• Severity levels are shared between PM and engineering with release-blocking semantics.
• Regressions are tagged so they can be counted per release.

7. Internal Testing Tracks

• Apple TestFlight: internal testers + external testers (up to 10,000). Review can take up to 24 hours for external.
• Play Store Internal Testing: internal testers (up to 100 emails), promotes to Closed/Open testing.
• Treat the internal track as the last mile. Do not submit to production from a build that has not been on internal for at least a few hours.

8. Support Loop

Close the loop with customer support:

• Weekly triage of user-reported issues against current release.
• Named owner for each repeatable issue.
• Use in-app feedback with device metadata (anonymized) to cut repro time.

9. Anti-Patterns

• QA gate at the very end. Bugs found then are 10x more expensive.
• Manual regression that takes four days. Trim ruthlessly or automate.
• Bug bashes without triage; they demoralize.
• Relying solely on emulators and simulators. Physical devices reveal what they cannot.
• Releasing on Friday without a rollback plan.

10. Metrics

Track per release:

• Time-to-first-bug-found in exploratory.
• Escaped defects (post-release bugs with severity above Low).
• Regression rate (new bugs in areas that passed last release).
• Crash-free sessions delta vs prior release.
• Store review latency.

Aim for downward trends. Trends matter more than absolute numbers.

Checklist

• A written release checklist is version-controlled in the repo.
• A regression suite covers flows automation misses and runs under 60 minutes per platform.
• A bug bash is scheduled for every release candidate.
• Real-world conditions (network, battery, locale) are explicitly tested.
• Internal testing track is used before any production submission.
• Support issues are triaged weekly and feed regression updates.
• Severity levels are shared across engineering and product.
• Per-release metrics are tracked and reviewed.