Bump Android Version

Bump Android Version

Overview

Drive an Android-version bump in Maestro commit-by-commit on a single feature branch: edit gradle, rebuild the on-device driver APKs, push a draft PR, dispatch test-e2e.yaml against the new system image, watch the test-android job, and diagnose+fix any newly-failing passing/ flows. Loop until green.

When to Use

• User asks to bump Maestro's Android compileSdk / targetSdk (e.g. "bump to API 36").
• User wants to validate Maestro against a new Android system image.

One commit per logical step. Don't bundle the gradle bump and the APK rebuild — they need to be separately revertible. After a failing run, fixes are committed and the workflow re-dispatched. Loop until green; do not stop on the first re-dispatch.

The loop only acts on passing/ failures. tests/demo_app/passing/ is the regression-detection suite — its flows are expected to pass, so a failure there is a real signal. tests/demo_app/failing/ is the negative-path suite (its flows are expected to fail) and is ignored end-to-end. The diagnose agent rejects failing/ artifacts and rejects retry-recovered flows in passing/ (see .claude/agents/diagnose-maestro-failure.md). If the run is otherwise green except for failing/, that's a green run.

Pre-flight

• Working tree clean (git status).
• main up to date: git fetch origin && git checkout main && git pull --ff-only origin main.
• The workflow's workflow_dispatch inputs android_version (and optional app / flow) must exist on the branch.
• android_version is a choice input — its options live in .github/workflows/test-e2e.yaml. If <new> isn't already listed (e.g. bumping to API 37), add it to the options: list in a separate commit before dispatching, otherwise the workflow rejects the input.
• The validate-inputs job rejects android_version <= android-29. Bumps below API 29 aren't supported.
• Branch name: bump-android-api-<old>-<new> (e.g. bump-android-api-34-36).

git checkout -b bump-android-api-<old>-<new> origin/main

Commit 1: bump compile/target SDK

Read current values from maestro-android/build.gradle.kts (compileSdk and targetSdk). Tell the user the current values verbatim, then ask for the new target:

Current compileSdk / targetSdk = <N>. What's the new API level?

Edit both lines. Also rg -n "compileSdk\s*=|targetSdk\s*=" --type kotlin --type-add 'kotlin:*.kts' to catch consistent occurrences in other modules — only update those that intentionally track the same SDK. Commit:

git add maestro-android/build.gradle.kts <other touched files>
git commit -m "chore(android): bump compile/target SDK from <old> to <new>"

Commit 2: rebuild driver APKs

./gradlew :maestro-android:assemble :maestro-android:assembleAndroidTest

The build's copyMaestroAndroid / copyMaestroServer finalizers update three checked-in files. Stage and commit only those:

git add maestro-client/src/main/resources/maestro-app.apk \
        maestro-client/src/main/resources/maestro-server.apk \
        maestro-client/src/main/resources/maestro-android-source.sha256
git commit -m "chore(android): rebuild driver APKs against API <new>"

If assemble fails, stop and surface the failure to the user with the gradle error and a suggested fix. Do not proceed without rebuilt driver APKs. Once we have assemble fixed and new drivers are commited, we can proceed to the next step.

Step 3: push draft PR + dispatch

git push -u origin bump-android-api-<old>-<new>
gh pr create --draft \
  --title "chore(android): bump compile/target SDK to <new>" \
  --body "Validates Maestro against API <new>. Driver APKs rebuilt. Test-e2e dispatched with system-images;android-<new>;google_apis;x86_64."

gh workflow run test-e2e.yaml --ref bump-android-api-<old>-<new> \
  -f android_version="android-<new>"

# capture the run id
sleep 3
gh run list --workflow test-e2e.yaml --branch bump-android-api-<old>-<new> --limit 1 --json databaseId,status

Always pass -f android_version=android-<new> — the default is android-32 and would silently re-test the old API. The workflow constructs the full system-images;... string internally. app and flow are optional narrowing knobs (defaults: demo_app, all flows); leave them off for the full bump-validation run.

Step 4: watch the test-android job

gh run watch <run_id>

Block in the foreground; the run is the bottleneck. Green here means every passing/ flow passed (terminally — retry-recovered counts as passed). failing/ outcomes do not affect the verdict. On green: update PR from draft → ready, done.

On red, download artifacts:

gh run download <run_id> --name maestro-root-dir-android -D /tmp/maestro-android-<run_id>

Step 5: diagnose (delegated to subagent)

Dispatch the diagnose-maestro-failure subagent with the artifact directory /tmp/maestro-android-<run_id> (or the GHA run URL — the subagent handles both). It diagnoses each passing-suite failure in isolation and returns a structured report grouped by root cause (cascades collapsed). It does not apply fixes — that's your job here.

When the subagent returns:

Fixes go in Maestro source — not in .github/workflows/test-e2e.yaml. A failing flow on a new API level reflects a behaviour Maestro users will also hit on their own machines/CI, not just ours. Patching the GHA workflow (e.g. extra adb shell settings put …, command-line tweaks, AVD setup steps) hides the regression from real users and ships a Maestro that's only green inside our CI.

Acceptable fix targets and their roles are documented in AGENTS.md (modules maestro-android/, maestro-client/, maestro-orchestra/, e2e/demo_app/). Driver-behaviour fixes belong in maestro-android/ or maestro-client/; fixture-only issues belong in e2e/demo_app/. Editing the driver source means rebuilding the driver APKs (./gradlew :maestro-android:assemble :maestro-android:assembleAndroidTest) before re-dispatching.

Valid edits to test-e2e.yaml during this loop are workflow-shape changes (matrix, retention, dispatch inputs) — and the narrow exception below. Anything else (e.g. extra adb shell settings put … to mask a Maestro driver gap) is off-limits. If the subagent proposes a workflow patch, push back: ask it (or yourself) where the same fix would live in maestro-android/ or maestro-client/ so users on the new API level inherit it automatically.

Exception — third-party app first-run UI not triggered by a Maestro API. A pre-installed app's own onboarding (Chrome Welcome / "Make Chrome your own", browser default-app picker, Play Protect prompts that fire at app launch) is environment-harness state, not driver behaviour. Maestro doesn't expose a public API that triggers them — they're side effects of the OS image we picked. These may be pre-disabled in the AVD setup step (typical knobs: setprop debug.chrome.command_line, settings put global … flags). Test: if Maestro called nothing related to this dialog, would it still appear? If yes → CI workaround is fair game. If no (e.g. setLocation triggers GMS Location Accuracy via FusedLocationProviderClient inside the driver) → fix the driver instead.

1. For each root cause in its report, present the proposed diff to the user (file path + one-line summary + diff + side effects). Reject any proposal that targets test-e2e.yaml for driver-behaviour reasons before showing it — re-scope to Maestro source first.
2. Ask consent explicitly per fix:

   Proposed fix for <root cause>: <one-liner>. Apply?

3. On approval: edit, commit with a focused message — don't bundle unrelated fixes.

   git commit -m "fix(android-<new>): <summary>"
   

4. On rejection: skip that fix; record what was rejected so you don't re-propose it next iteration.

After all approved fixes for this iteration are committed:

• If any fix was applied: push and re-dispatch, then loop back to Step 4 with the new run_id.

  git push
  gh workflow run test-e2e.yaml --ref bump-android-api-<old>-<new> \
    -f android_version="android-<new>"
  

• If no fixes were applied (everything was out-of-scope, or the user rejected every proposal): pause and ask the user how to proceed. Do not keep dispatching.

Termination

Stop when test-android is green on the latest run. Then:

gh pr ready <pr_number>

If the loop has gone three iterations without progress (same flow keeps failing for different reasons after fixes), pause and ask the user how to proceed — don't spin indefinitely.

Anti-patterns

• Bundling gradle + APK rebuild into one commit — kills bisectability. Always two commits.
• Dispatching with default inputs — android_version defaults to android-32 and hits the old API. Always pass -f android_version=android-<new>.
• Forgetting to extend the android_version choice enum — the workflow rejects API levels not listed in options:. Add <new> to the enum in a separate commit before the first dispatch.
• Auto-applying Maestro source fixes without consent — every Kotlin patch needs explicit user approval first.
• Patching .github/workflows/test-e2e.yaml to mask a driver behaviour gap — workflow band-aids hide the regression from users running Maestro outside our CI. Fix maestro-android/ or e2e/demo_app/ instead so the fix ships with the driver APKs.
• Treating failing/ artifacts as regressions — that suite is expected to fail.
• Treating ❌ screenshots in passing/ as regressions on their own — when retryCommand recovers a failed attempt, Maestro writes only the final COMPLETED entry to commands-(<flow>).json but leaves the ❌ screenshot from the failed attempt on disk (e.g. screenshot-❌-<ts>-(retry).png for the retry flow itself). The diagnose agent uses commands-*.json containing a terminal FAILED entry as the source of truth — orphan ❌ screenshots without a matching FAILED entry are noise.
• Committing on main — always work on the bump branch. Verify with git status before every commit.
• Skipping the screenshot read — the screenshot tells you what maestro.log and the JSON cannot. It's the highest-signal artifact. Always read it.