Smoke Test Analyzer

smoke-test-analyzer

⚙️ Read modules/config-loader.md first. If mode = markdown-only, output reports to local .md instead of Google Sheet.

💡 第一次聽到 T0/T1/T2/T3 分層？ 先看 concept-zh.md 中文入門導讀（5 分鐘搞懂「為什麼不該每次 PR 都跑全部測試」）。

Analyze existing automated tests to curate the optimal Daily Smoke subset for CI. Supports iOS and Android.

Why This Matters

Running all tests on every CI trigger is wasteful. A well-tiered test suite catches 90% of regressions in 10% of the time by running the right tests at the right frequency.

Test Tiers

Time budgets adjustable via config.smoke.time_budgets.

Platform Detection

Auto-detect based on project files:

• iOS: *.xcodeproj, {ProjectName}Tests/, {ProjectName}UITests/, *.xctestplan
• Android: build.gradle(.kts), src/test/, src/androidTest/

If both detected, analyze both and produce a cross-platform summary.

可從 config.platforms.{ios,android}.test_dirs 自訂掃描目錄。

4-Phase Execution

Phase 1: Scan & Inventory

Collect all test files and count methods per file.

iOS: Scan {ProjectName}Tests/, {ProjectName}UITests/, internal Swift package Tests/ dirs. Count @Test and func test*. Read existing *.xctestplan to understand current CI membership.

Android: Scan app/src/test/, app/src/androidTest/. Count @Test. Check Gradle test tasks and CI workflows for current configuration.

Output an inventory table:

Phase 2: Smoke Suitability Scoring

Score each test file on 5 weighted criteria (1-5 scale). See references/scoring-heuristics.md for detailed signals and examples.

Tier assignment by score:

• = 4.0 → T0 | 3.0-3.9 → T1 | 2.0-2.9 → T2 | < 2.0 → T3

Phase 3: Classification & Recommendation

Present scored results grouped by tier, per platform. Include:

• T0/T1/T2/T3 tables with file, methods, score, rationale
• Cross-platform summary (tier counts + method counts per platform)
• Flaky risk report (flagged tests with issue + fix suggestion)

Ask user to review and adjust before generating configs.

Phase 4: Generate CI Configuration

Based on user-confirmed tiers, generate platform-specific configs.

• iOS: see references/ios-config.md for .xctestplan format and CI workflow snippets
• Android: see references/android-config.md for Gradle filter / @Tag / Suite options

Output per platform:

1. CI test configuration file (.xctestplan or Gradle config)
2. CI workflow snippet (GitHub Actions yaml)
3. Flaky test fix recommendations

If both platforms, also generate a Google Sheet (4 tabs: Summary / iOS Detail / Android Detail / Flaky Report). Upload target: {{GDRIVE_QA_FOLDER_ID}}/CI Smoke Plan/.

Markdown-only mode: write .claude/testing/ci-smoke/report.md and per-platform *.md reports instead of Sheet.

Quality Checks

• Every T0 test must be deterministic (no sleep, no timing dependency)
• T0+T1 total < 10 min per platform (依 config.smoke.time_budgets)
• No T0 test depends on network or external service
• Existing CI coverage not regressed (T2 tests still run somewhere)
• Same core business flows covered on both platforms

When to Re-run

• New test files added (monthly or per-feature)
• Flaky tests identified in CI logs
• CI execution time exceeds budget
• After test-automation generates new scripts

Config Dependencies