# E2e Pr Stabilizer > Stabilizes or optimizes Playwright E2E tests on one PR via a local-first loop, then ratifies with a single CI run. Pulls Dash0 spans (`git.pull_request_link`) as the historical baseline, then captures every iteration's evidence locally with `--trace=on` (same OTel exporter, same trace schema). Validation is empirical, not predictive: before commit, every new locator must resolve against source (static grep) or the live app (`locator.count()`); after commit, the fixed test must pass three consecutive local runs before the single push. Modes: `stabilize` (default) heals flaky / failing tests; `optimize` is report-only and ranks slow-action wins by measured ms saved. Refuses `.skip`, `.fixme`, `waitForTimeout`, or any check-weakening edit. Use when a PR has flaky or failing E2E tests or when you want to find slow tests worth tightening. Triggers on "stabilize this PR", "fix flaky e2e", "heal playwright on PR", "ui-e2e is failing", "self-heal e2e", "optimize e2e", "/e2e-pr-stabilizer". - Skill: `mthines/e2e-pr-stabilizer` (Agent Skill, multi-file: 11 files) - Install (CLI): `npx skillmds@latest add mthines/e2e-pr-stabilizer` - Raw SKILL.md: https://api.skillmd.com/api/skills/mthines/e2e-pr-stabilizer/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Docs & Writing - License: MIT - Author: mthines (https://skillmd.com/u/mthines) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/mthines/e2e-pr-stabilizer --- # E2E PR Stabilizer Stabilize the Playwright E2E suite for a single pull request using **evidence, not assumptions**. Spans, traces, and the live app are the source of truth — not the CI dashboard. This skill never proposes a fix without a measurement to point at, and never commits a fix until three consecutive local runs prove it works. > **This `SKILL.md` is a thin index.** > Detailed procedures live in [`rules/*.md`](./rules) and [`templates/*.md`](./templates). > Each phase loads only what it needs. --- ## What this skill combines | Source | Role | |--------|------| | [Playwright Healer agent](https://playwright.dev/docs/test-agents) — external; one of the [Playwright Test Agents](https://playwright.dev/docs/test-agents) on the [Playwright MCP server](https://github.com/microsoft/playwright-mcp) | Test-debugging methodology — how to fix a Playwright test correctly. | | [`/playwright-trace-analyzer`](../../analysis/playwright-trace-analyzer/SKILL.md) | Per-run `trace.zip` extraction, hotspot ranking. | | [`/ci-auto-fix`](../../delivery/ci-auto-fix/SKILL.md) | Reused only for Phase 7's single push + watch — the iteration loop no longer lives here. | | Dash0 MCP server (`dash0-dev` or `dash0-prod`) | Historical evidence — failure recurrence, retry counts, span-level evidence across CI runs. | | Local Playwright runner | Primary evidence source — trace.zip per run, OTel spans to Dash0 (`ci.is_ci=false`), and the live app for selector verification. | | GitHub Actions (one call) | Final CI ratification at Phase 7. | This skill is the orchestrator over those. > **External dependency — the Playwright Healer agent.** The healer methodology > this skill drives fixes through is **not** an agent in this repo; it is > Playwright's own **healer** ([Playwright Test Agents](https://playwright.dev/docs/test-agents): > planner / generator / healer) running on the > [Playwright MCP server](https://github.com/microsoft/playwright-mcp). Set it up > with `npx playwright init-agents --loop=claude` (Playwright ≥ 1.56). At runtime, > Phase 5 **uses the healer when the Playwright MCP is connected** (`mcp__playwright__*` > tools present) and **falls back to the inline root-cause methodology** when it > isn't — so the skill works with or without it, but is strongest with it. It does not duplicate their content — each phase delegates. --- ## Modes | Mode | Default | Entry rule (what enters the fix queue) | Phase 5 (edits) | Phase 6 (local 3-pass gate) | Phase 7 (CI ratification) | Phase 8 output | |------|---------|----------------------------------------|-----------------|-----------------------------|---------------------------|----------------| | `stabilize` | **yes** | `failure_rate ≥ 0.10` over ≥ 5 attempts, or `flake_count ≥ 2`. | Drafted, then double-gated before commit. | Required — 3 consecutive local passes per fixed test. | One push, one CI watch. | Stabilization report with before / after numbers, local-pass log, CI verdict. | | `optimize` | | Top-N slowest tests by total time, or actions with `dur > 5×median`. | **Skipped.** | **Skipped.** | **Skipped.** | Recommendations-only report — humans apply the wins. | `stabilize` is the default because optimization edits (tightening timeouts, removing waits) carry flake risk that warrants human judgment. `optimize` runs Phases 1–4 only and emits a ranked recommendations report. ## Input `$ARGUMENTS` is parsed as `[mode] [pr-ref]` in any order: - `optimize` (literal token) selects optimize mode; anything else is treated as `pr-ref`. - `pr-ref` is a PR URL (`https://github.com///pull/13319`) or PR number (`13319`). - If `pr-ref` is missing, auto-detect the open PR for the current branch (same path as [`/ci-auto-fix`](../../delivery/ci-auto-fix/SKILL.md) Step 0). - If `mode` is missing, default to `stabilize`. Resolve mode + PR before doing anything else. See [`rules/input-resolution.md`](./rules/input-resolution.md). --- ## Workflow Eight phases. Do not skip a gate. Phases 5, 6, and 7 are skipped in `optimize` mode (the `Modes` column says so explicitly). | Phase | Name | Modes | Rule file | Gate | |-------|------|-------|-----------|------| | 0 | Resolve target | both | [`rules/input-resolution.md`](./rules/input-resolution.md) | Mode + PR URL + branch + head SHA + owner / repo printed. | | 1 | Pull historical telemetry | both | [`rules/telemetry-driven-analysis.md`](./rules/telemetry-driven-analysis.md) | Dash0 spans for this PR fetched and grouped by test name; failure recurrence + retry counts measured (stabilize) **or** action `dur` distribution measured (optimize). | | 2 | Local reproduction + trace capture | both | [`rules/local-iteration.md`](./rules/local-iteration.md) | Each queued test run locally with `--trace=on`; trace.zip + (where available) fresh Dash0 spans tagged `ci.is_ci=false` captured. | | 3 | Correlate spans ↔ traces | both | [`rules/root-cause-and-fix.md`](./rules/root-cause-and-fix.md) | Each queued test has a span-side signature **and** a trace-side hotspot. | | 4 | Root-cause synthesis | both | [`rules/root-cause-and-fix.md`](./rules/root-cause-and-fix.md) | A single, evidence-anchored hypothesis per candidate, citing the span signature and the trace hotspot. Speculative hypotheses become `recommendation-only` entries, not fixes. | | 5 | Draft fix + selector-existence check | **stabilize only** | [`rules/root-cause-and-fix.md`](./rules/root-cause-and-fix.md), [`rules/fix-validation.md`](./rules/fix-validation.md), [`rules/guard-rails.md`](./rules/guard-rails.md) | Diff drafted; every new locator proven to resolve against source (static grep) **or** the live app (`locator.count() ≥ 1`). A locator that fails both checks is hallucinated — discard the diff and re-enter Phase 4 with that evidence. | | 6 | Local verification — 3 consecutive passes | **stabilize only** | [`rules/local-iteration.md`](./rules/local-iteration.md) | Fixed test runs locally ≥ 3 times with `--trace=on` and passes **3 times in a row**. A single failure or flake within the streak resets the counter. Maximum 10 attempts per test before escalating. | | 7 | CI ratification — one push, one watch | **stabilize only** | [`rules/verification-loop.md`](./rules/verification-loop.md) | All passing fixes committed and pushed in a single push event; the resulting CI run is watched to conclusion and its telemetry compared against the Phase 1 baseline. | | 8 | Report | both | [`templates/stabilization-report.md`](./templates/stabilization-report.md) | Stabilize: report with before / after numbers + local-pass log + CI verdict + residual risk. Optimize: recommendations-only report ranked by measured wall-clock impact. | Inner iteration in `stabilize` mode is local and bounded — see [`rules/local-iteration.md`](./rules/local-iteration.md). The CI step in Phase 7 runs **once**. If CI disagrees with the local result, that is a signal to escalate, not to re-enter the loop blindly. --- ## Required reading by phase Load on demand. Do not preload. | Phase | Files | |-------|-------| | 0 | [`rules/input-resolution.md`](./rules/input-resolution.md) | | 1 | [`rules/telemetry-driven-analysis.md`](./rules/telemetry-driven-analysis.md), [`references/dash0-mcp-filters.md`](./references/dash0-mcp-filters.md) | | 2 | [`rules/local-iteration.md`](./rules/local-iteration.md) | | 3–4 | [`rules/root-cause-and-fix.md`](./rules/root-cause-and-fix.md), [`rules/self-improvement-loop.md`](./rules/self-improvement-loop.md) (read lessons — stabilize only) | | 5 | [`rules/root-cause-and-fix.md`](./rules/root-cause-and-fix.md), [`rules/fix-validation.md`](./rules/fix-validation.md), [`rules/guard-rails.md`](./rules/guard-rails.md) | | 6 | [`rules/local-iteration.md`](./rules/local-iteration.md) | | 7 | [`rules/verification-loop.md`](./rules/verification-loop.md), [`rules/self-improvement-loop.md`](./rules/self-improvement-loop.md) (write lessons on ratification — stabilize only) | | 8 | [`templates/stabilization-report.md`](./templates/stabilization-report.md) | For trace mechanics (zip → JSONL → action timeline), defer to [`/playwright-trace-analyzer`](../../analysis/playwright-trace-analyzer/SKILL.md). Do **not** re-implement. --- ## Core principles 1. **Iterate locally, ratify on CI.** The inner loop is local because it is seconds-per-run, has the same trace artifacts, and emits the same OTel spans. CI runs once at the end as ground truth. 2. **Data first, hypothesis second.** Every fix is anchored to (a) a span with a measured failure rate, or (b) a trace action with a measured `dur`. "I think this is flaky" is not a finding. 3. **Two evidence layers, not one.** Spans tell you *which tests fail and how often across runs* (historical and local). Traces tell you *why one specific run failed*. A fix is only credible when both layers agree. 4. **Validate empirically, not predictively.** Confidence in a fix comes from running it, not from scoring it on paper. The two gates the skill enforces are both deterministic: selectors must exist (verifiable against source or live app), and the fixed test must pass three consecutive local runs (verifiable from exit codes). 5. **Selectors must exist before they are used.** Every new locator in a draft fix is verified against the component source (static) and, when ambiguous, the running app (live). The skill never commits a fix that names an element that does not exist. 6. **Three consecutive local passes or no commit.** A single passing run can be a coin flip on a flake. Three in a row — hardened with `--repeat-each=3` when the measured flake rate is low (see [`rules/local-iteration.md`](./rules/local-iteration.md)) — filters out most unfixed flakes before we spend a CI cycle. The streak is necessary but not sufficient evidence; Phase 7's CI ratification plus the telemetry comparison is the real confirmation. 7. **Never weaken the suite.** No `.skip`, `.fixme`, `waitForTimeout`, `continue-on-error`, `--no-verify`, or removed assertions. The full list lives in [`rules/guard-rails.md`](./rules/guard-rails.md). 8. **Test-side fix unless the trace proves otherwise.** Most flakes are selector, timing, or state-management bugs in tests. If the trace evidence points to product code, surface it to the user as a separate recommendation — do not silently mutate app code. 9. **One PR at a time.** Cross-PR refactors belong in a different skill. --- ## Self-Improvement `/e2e-pr-stabilizer` gets better across runs through a two-tier lessons loop (fast episodic tier + gated promotion), like `autonomous-workflow` and `fix-bug`. In **`stabilize` mode** it **reads** `e2e-pr-stabilizer-lessons` at Phase 4 (biasing the P1–P6 pattern classification and the Phase 5 locator strategy) and **writes** at Phase 7 — gated on the **telemetry ratification signal**, not the local 3-pass streak, so only fixes that CI actually confirmed `fixed` accrue a working lesson. The two tiers split naturally: the `global` scope holds universal race-shape → fix-shape mappings, the `repo::{owner}/{repo}` scope holds app-specific locator robustness. Lessons are **advisory** — they never relax a guard-rail, an empirical gate, or the 3-consecutive-pass requirement. A recurring lesson (`seen_count >= 3`) is promotion-eligible via `/create-skill diagnose e2e-pr-stabilizer`. On the Phase 7 ratified verdict it also contributes a **flaky hotspot** to the shared **`codebase-knowledge`** bucket (`hotspot::` counter) — a fact no other host produces, so every later code-changer plans around a known-unstable file. Write only, merge-never-clobber, on the ratified SHA (contract: [`../../../agents/shared/rules/codebase-knowledge.md`](../../../agents/shared/rules/codebase-knowledge.md); mechanics in the loop file's cross-bucket write section). `optimize` mode skips the loop (no fix, no ratification signal). Lessons run through LoreKit's `memory.*` tools (the `lorekit-memory` skill); if LoreKit is not connected the loop is a silent no-op. Full contract: [`rules/self-improvement-loop.md`](./rules/self-improvement-loop.md). ## Anti-patterns One-liners; the full list lives in [`rules/guard-rails.md`](./rules/guard-rails.md). - Pushing a fix because "the diff looks right" without three consecutive local passes. - Trusting a single local pass as proof — flakes pass once routinely. - Drafting a fix that uses `getByTestId('foo')` when nothing in the component source emits `data-testid="foo"`. - Patching `waitForTimeout(1500)` to mask a race instead of fixing the wait condition. - Marking a test `.fixme()` because it is "flaky" without a measured cause. - Treating a single failed CI run as evidence — flakes are statistical, so fetch the span history. - Re-running CI hoping for a green without applying a code change. - Editing product code based on speculation when the trace points at a selector or test-state issue. --- ## Quickstart ```text /e2e-pr-stabilizer # stabilize, auto-detect PR /e2e-pr-stabilizer 13319 # stabilize PR 13319 /e2e-pr-stabilizer https://github.com///pull/13319 # stabilize via URL /e2e-pr-stabilizer optimize # optimize, auto-detect PR /e2e-pr-stabilizer optimize 13319 # optimize PR 13319 ``` Once invoked, the skill drives end-to-end: 1. Resolves the mode and the PR. 2. Queries the Dash0 MCP for E2E spans filtered to this PR (`git.pull_request_link`) — historical baseline. 3. **Reproduces locally** with `--trace=on`, capturing trace.zip and (where the local OTel reporter is wired) fresh spans. 4. Correlates and produces an evidence-anchored finding set. 5. **stabilize:** drafts each fix, verifies every new locator resolves against source or the live app, then commits locally. 6. **stabilize:** runs the fixed test locally until it passes 3 times in a row (per fix). 7. **stabilize:** pushes once; watches the CI run; compares fresh telemetry to baseline. 8. Emits the report — stabilization (before / after + local-pass log + CI verdict) or optimization (recommendations). --- ## Definition of Done ### Both modes - [ ] Mode (`stabilize` | `optimize`) and PR target resolved and printed. - [ ] Historical telemetry pulled from the Dash0 MCP using the documented filter set, grouped by test name. - [ ] Each queued test reproduced locally with `--trace=on`; trace artifacts captured per run. - [ ] Each candidate has a span-side signature and a trace-side hotspot. - [ ] Report written using the template, with the mode stated and findings ranked by measured impact. ### `stabilize` only - [ ] Every new locator in every applied fix was verified against the component source or via a live `locator.count() ≥ 1` probe. - [ ] Every applied fix passed 3 consecutive local runs with `--trace=on` and no failures or flakes within the streak. - [ ] Fixes committed locally, pushed in one push, CI run watched to conclusion. - [ ] Fresh telemetry pulled and compared to baseline — failures eliminated, retry counts reduced. - [ ] No `.skip`, `.fixme`, `waitForTimeout`, or `continue-on-error` introduced (guard-rails check passed). ### `optimize` only - [ ] No commits, no pushes, no edits to test files. - [ ] Each recommendation cites an estimated wall-clock saving (ms) based on the trace evidence.