Generate Uat

Generate UAT Test Plan

Purpose

Produces a client-ready UAT document from GitHub issue data stored locally. The document contains testable scenarios derived from Acceptance Criteria, enriched with test steps and scope notes extracted from ticket comments.

What you get:

• A markdown UAT test plan with one table per feature group
• Cross-platform deduplication: mirrored iOS/Android ACs are consolidated into single rows with per-platform result columns, reducing redundancy by ~30–40%
• Platform-specific scenarios clearly marked as "iOS only" or "Android only"
• A "Known Limitations and Issues" section combining permanent limitations, deferred items, and on-hold blockers in one table
• A "Tickets Without ACs" appendix for visibility into gaps
• A "Technical Verification" section for backend-only items clients cannot directly test

Core principle: Requirement purity. Every scenario must describe observable user behavior. No implementation details, no internal jargon, no developer tooling references. If a scenario cannot be made client-testable, it goes to Technical Verification.

Inputs

Two entry modes — ask the user which one applies:

Mode A: From Release Summary Report (default)

The user provides a path to a release-summary-by-feature-*.md report file generated by scripts/release_summary_by_feature.py.

This report contains markdown tables with columns: Platform, #, Title, Status, Type, Updated, Milestone, Assignees, Recent Activity.

The # column contains links in the format [#NNN](https://github.com/DBDHub/{slug}/issues/NNN) — extract the issue number and repo slug from these links.

Mode B: Manual Ticket List

The user provides a list of ticket numbers with their repo identifiers. Example formats to accept:

• iOS #982, Android #1050, Backend #259
• A markdown table or bullet list with numbers and platforms

Phase 1: Setup and Selection

Step 1 — Determine input mode

Use AskQuestion:

id: input_mode
prompt: How would you like to provide tickets for the UAT plan?
options:
  - id: report
    label: From a release-summary-by-feature report file
  - id: manual
    label: I'll provide a list of ticket numbers

If report: ask the user for the file path. Read the file and parse all feature group sections.

If manual: ask the user for their ticket list. Parse ticket numbers and platform/repo identifiers. Skip to Phase 2.

Step 2 — Parse the report

From the release-summary-by-feature report, extract:

• Feature group names (level-2 headings like ## 🔔 Push Notifications)
• For each table row: Platform, issue number, title, status, URL

Build an internal index:

{feature_group} → [{platform, number, title, status, repo_slug, url}, ...]

Derive repo_slug from the URL path:

• SecuritasOfficer-iOS → platform iOS, folder DBDHub-SecuritasOfficer-iOS
• SecuritasOfficer-Android → platform Android, folder DBDHub-SecuritasOfficer-Android
• sna_wfm_api → platform Backend, folder DBDHub-sna_wfm_api

Step 3 — Select feature groups

Use AskQuestion with allow_multiple: true:

id: feature_groups
prompt: Which feature groups should be included in the UAT plan?
options: [one option per parsed feature group, using the group name as label]

Step 4 — Select statuses to include

Use AskQuestion with allow_multiple: true:

id: statuses
prompt: Which board statuses should be included? (Only tickets in these statuses will appear in the UAT plan)
options:
  - id: sr     | label: Securitas Review
  - id: qa     | label: QA - Functional Complete
  - id: review | label: In Review - Unit Complete
  - id: passed | label: Passed
  - id: ip     | label: In Progress
  - id: todo   | label: Todo DEV
  - id: hold   | label: On Hold
  - id: spike  | label: Spike
  - id: backlog| label: Backlog

Step 5 — Confirm selection

Display the filtered ticket list to the user:

• Group by feature group
• Show ticket number, platform, title, status
• Show total count

Ask the user to confirm before proceeding. Example: "This will generate UAT scenarios for N tickets across M feature groups. Proceed?"

Phase 2: Data Extraction

For each selected ticket, read the issue markdown file at:

{workspace}/Github-Issue-Extractor/issues/{repo-folder}/issue-{number}.md

Where {repo-folder} is derived from the platform:

• iOS → DBDHub-SecuritasOfficer-iOS
• Android → DBDHub-SecuritasOfficer-Android
• Backend → DBDHub-sna_wfm_api

What to extract from each issue file

2a. Acceptance Criteria (primary source of test scenarios)

Look for these headings (in order of precedence):

1. ## ✅ Acceptance Criteria
2. ## Acceptance Criteria
3. ## ✅ Success Criteria (spikes)

Within the AC section, extract each AC block. AC blocks follow these patterns:

Pattern 1 — Bold title + BDD steps (most common):

**AC1: Title Here**
Given [precondition]
When [action]
- Then [outcome]
- And [additional outcome]

Pattern 2 — Heading-based ACs:

### AC1: Title Here
[bullet points or BDD steps]

Pattern 3 — Checkbox list (spikes / success criteria):

- [ ] Condition to validate
  - [ ] Sub-condition

For each AC, capture: AC identifier (e.g. "AC1"), title, and the full body (Given/When/Then or bullet points).

2b. Context (for pre-conditions and feature description)

Look for these headings:

1. ## 📌 Context / Background
2. ## Context
3. ## User Statement

Extract a brief summary (first 2-3 sentences) for the UAT pre-conditions section.

2c. Comments — Test Steps and QA Notes

Under ## Comments, each comment has the format:

### Comment by {username} on {ISO-timestamp}

{comment body}

Scan comment bodies for testing-relevant content:

• Numbered step lists (e.g. "1. Open the app 2. Navigate to...")
• Phrases indicating test procedures: "steps to reproduce", "how to test", "verification", "tested scenario", "precondition"
• QA verification results with checkmarks or status markers
• Specific scenario descriptions with outcomes

When found, associate these steps with the relevant ACs they describe.

2d. Comments — Scope and Deferral Notes

Scan comments for scope-limiting language:

• "deferred", "next release", "out of scope", "not included in this release"
• "will be available in", "postponed", "V2", "future", "blocked"
• "limitation", "known issue", "not supported yet"

False-positive guard: After finding a match, read the full comment in context. Reject matches where:

• The comment is restoring the ticket to scope (e.g. "moving back to [status]", "re-scoped for [current release]", "adding back")
• The language describes a past state that has since been resolved (e.g. "was not released in Jan" followed by "now in scope for April")
• The deferral language refers to a different ticket or feature, not the one being extracted

Only record a deferral if the comment clearly states that a specific capability or sub-feature will not be available in the current release.

Also check for a ## Limitations or ## Limitations (Documented) section in the issue body.

Record each deferral note with: ticket number, the deferred item description, type (Limitation/Deferred/On Hold), and the source (comment author + date, or Limitations section).

2e. Frontmatter metadata

Extract from YAML frontmatter: title, number, board_status, milestone, assignees, url, labels.

2f. Defect ticket handling (when no ACs are found)

If a ticket has no formal Acceptance Criteria section (2a returns nothing), check whether it is a defect:

Detection: A ticket is a defect if any of these are true:

• Labels include "bug", "defect", "fix", or "QA"
• Title contains patterns like "incorrect", "missing", "wrong", "broken", "issue", "fix", "not working", "not triggered", "duplicate", "alignment"
• Issue body contains "Steps to Reproduce", "Expected Behavior", "Actual Behavior", or "Bug Description"

Extraction from defect tickets:

Look for these sections in the issue body (in order of precedence):

1. Steps to Reproduce / How to Reproduce → maps to the Steps column
2. Expected Behavior / Expected Result → maps to Expected Result column
3. Actual Behavior / Current Behavior → context for the scenario description (what was wrong)
4. Screenshots / visual references → note in scenario description if the fix is visual

If none of these sections exist, use the issue title and body description to infer:

• Steps: derive from the title and context (e.g. "French translation alignment issues" → set language to French, navigate to the affected screen)
• Expected Result: the defect should no longer occur (e.g. "Text is properly aligned and does not overflow")

Classification — client-testable vs technical:

After extracting, classify the defect:

• Client-testable (goes to main Test Scenarios table): The fix is visible in the UI — visual/layout issues, navigation bugs, text/translation problems, interaction bugs, missing UI elements. The tester can verify by looking at the screen or interacting with the app.
• Technical (goes to Technical Verification appendix): The fix is in backend logic, event logging, database operations, or API responses. The tester cannot verify without developer tools.

Scenario generation for client-testable defects:

Generate one scenario row per defect with:

• Scenario: "Verify [plain-language description of what was fixed]" — framed as confirmation the fix works, not as a bug report
• Steps: from repro steps or inferred from context
• Expected Result: the correct behavior (from "Expected Behavior" section, or inferred)

All purity and clarity rules apply. Defect scenarios go through the same Phase 4 (Purity) and Phase 4.5 (Clarity) checks as AC-derived scenarios.

Cross-platform deduplication (Phase 2.5)

After extracting ACs from all tickets, identify mirrored iOS/Android ticket pairs within each feature group:

1. Match ticket pairs by comparing:

   • Title patterns: strip platform prefix (e.g. "iOS -- " or "Android -- ") and compare the remainder
   • Parent epic reference in frontmatter (if parent_issue is present)
   • AC titles: if both tickets have ACs with the same identifiers (AC1, AC2, etc.) and similar titles

2. Compare ACs within matched pairs:

   • For each AC in the iOS ticket, find the corresponding AC in the Android ticket (by AC number and/or title match)
   • If the Given/When/Then content is substantively identical (ignoring platform name substitutions like "iOS" → "Android"), mark the AC pair as duplicate
   • If the content differs materially (e.g. a locale constraint, platform-specific UI behavior), mark it as platform-specific

3. Build a consolidated scenario list:

   • Duplicate AC pair → single row with Platform = "iOS, Android", both ticket links, one set of Steps/Expected Result
   • Platform-specific AC → separate row with Platform = "iOS only" or "Android only", the other ticket/result columns show "—"
   • Unpaired ticket (only one platform has it) → single row for that platform only

This deduplication typically reduces the scenario count by 30–40% for feature groups with full platform mirrors.

Phase 3: UAT Document Generation

Document structure

# UAT Test Plan — [Feature Group 1], [Feature Group 2], ...

**Generated:** [today's date]
**Source:** [report filename or "Manual ticket selection"]
**Platforms:** [platforms present in selection]
**Total scenarios:** [count]

---

## How to Use This Document

1. Execute each test scenario on **every platform listed** in the Platform column
2. Record Pass (P) or Fail (F) in the Result column — one result per platform
3. For failures, note the actual behavior observed and the platform affected
4. Backend scenarios marked "Technical Verification" require developer assistance

---

## [Feature Group Name]

**[N] scenarios** across [platforms list]

### Pre-conditions
- [Common setup derived from Context sections of tickets in this group]

### Test Scenarios

| # | Platform | iOS Ticket | Android Ticket | Scenario | Steps | Expected Result | Result (iOS) | Result (Android) |
|---|----------|------------|----------------|----------|-------|-----------------|--------------|------------------|
| 1 | iOS, Android | [#NNN](url) | [#NNN](url) | [scenario name from AC] | 1. [step]<br>2. [step]<br>3. [step] | [observable outcome] | | |
| 2 | iOS only | [#NNN](url) | — | [platform-specific scenario] | 1. [step]<br>2. [step] | [observable outcome] | | — |

### Known Limitations and Issues

| # | Type | Platform | Description | Reference |
|---|------|----------|-------------|-----------|
| 1 | Limitation | iOS | [permanent restriction or accepted behavior] | [#NNN](url) |
| 2 | Deferred | Android | [will be addressed in a future release] | [#NNN](url) |
| 3 | On Hold | iOS, Android | [blocked by external dependency] | [#NNN](url) |

Types:
- **Limitation** — permanent (OS restriction, accepted design decision, V1 trade-off)
- **Deferred** — will be addressed in a future release (has a tracking ticket)
- **On Hold** — blocked by an external dependency (e.g. third-party defect)

---

[Repeat for each feature group]

---

## Appendix A: Tickets Without Testable Content

These tickets had no Acceptance Criteria AND no reproduction steps or testable description. Most tickets without formal ACs are defects — those should be processed into the main Test Scenarios table (if client-testable) or Technical Verification (if backend-only). Only truly untestable tickets belong here.

| Ticket | Platform | Title | Status | Notes |
|--------|----------|-------|--------|-------|
| [#NNN](url) | iOS | [title] | [status] | [any relevant context] |

---

## Appendix B: Technical Verification (Backend)

Backend scenarios that require developer or technical QA involvement. Includes both backend-only stories and technical defects that cannot be verified through the UI.

| # | Ticket | Scenario | Verification Method | Expected Result | Result |
|---|--------|----------|--------------------|-----------------|---------| 

Table generation rules

1. One AC = one or more scenario rows. If an AC describes multiple distinct conditions (e.g. AC5 with "returns to geofence AND leaves again AND no cap"), split into separate rows.

2. Cross-platform deduplication (important). iOS and Android often have mirrored stories with identical ACs. Before generating rows, compare ACs across platforms within the same feature group:

   • Match by AC title and content, not by ticket number. Two ACs are duplicates if their Given/When/Then steps describe the same user behavior on different platforms.
   • If ACs are identical: merge into a single row with Platform = "iOS, Android", show both ticket numbers (iOS Ticket + Android Ticket columns), and provide separate Result columns per platform.
   • If ACs differ in a material way (e.g. a locale not available on one platform, a platform-specific UI behavior): keep them as separate rows with Platform = "iOS only" or "Android only" and mark the other platform's Result column with "—".
   • If only one platform has the ticket (no mirror): single row with that platform only, "—" in the other ticket/result column.

3. Table columns for consolidated format:

   • # — scenario number
   • Platform — "iOS, Android", "iOS only", or "Android only"
   • iOS Ticket — link to iOS issue, or "—"
   • Android Ticket — link to Android issue, or "—"
   • Scenario — AC-derived scenario description (full sentence, see Phase 4.5 Clarity Check)
   • Steps — numbered action steps
   • Expected Result — observable outcome
   • Result (iOS) — tester fills in P/F
   • Result (Android) — tester fills in P/F, or "—" if not applicable

4. Steps column: Populate from:

   • Given/When/Then steps in the AC (rewrite as numbered action steps)
   • Test steps found in comments (if they map to this AC)
   • If no steps are available, write "See scenario description" and include the AC conditions in the Scenario column
   • Formatting: Use <br> between numbered steps so each step renders on its own line within the table cell (e.g. 1. Step one<br>2. Step two<br>3. Step three)
   • Self-contained: Every scenario's Steps must include the full setup to reach the testable state. Do not use shorthand like "Trigger X" or "View the Y screen" without spelling out how. Do not reference other scenarios (e.g. "Repeat scenarios 15–16"). A tester must be able to execute any single row without having read any other row — the only shared context is the section Pre-conditions block.

5. Expected Result column: Derived from the "Then" clauses of ACs. Must be:

   • Observable (user can see/hear/measure it)
   • Specific (no "works correctly")
   • Pass/fail deterministic

6. Sorting: Within each feature group table:

   • Primary: Scenario logical order (by AC number of the iOS ticket, or Android if iOS is absent)
   • Secondary: Platform-specific rows after their shared counterpart

7. Backend tickets: If a backend ticket has no user-observable behavior (pure API/infrastructure), move its scenarios to Appendix C: Technical Verification.

8. How to detect mirrored tickets: Within a feature group, match iOS and Android tickets that share:

   • Similar title patterns (e.g. "iOS -- Push Notifications -- Off-Site Alert" and "Android -- Push Notifications -- Off-Site Alert")
   • The same parent epic (if parent_issue is present in frontmatter)
   • Identical or near-identical AC titles (e.g. both have "AC1: Single Share CTA") When in doubt, compare the AC bodies — if the Given/When/Then are substantively the same (ignoring platform name), they are mirrors.

Phase 4: Purity Check

Before finalizing the document, run a self-check pass on every scenario row. This is modeled on the requirement purity and testability checks from /validate-requirements.

Purity rules

Applying purity

For each scenario row:

1. Check Steps and Expected Result columns against all 7 rules
2. Rewrite any violations in place
3. If a scenario cannot be made pure (e.g. purely technical backend validation), move it to Appendix C
4. Track the count of rewrites for the summary

Purity exceptions

Some technical terms are acceptable when they are part of the user-facing experience:

• "push notification" (users see these)
• "location services" / "location permissions" (users interact with these)
• "clock in" / "clock out" (domain terminology the client uses)
• "geofence" — rewrite as "location boundary" unless the client uses "geofence" in their own documentation

Phase 4.5: Scenario Clarity Check

After purity, run a clarity pass on every scenario row. The target reader is a UAT tester who may be encountering this feature for the first time. They may not be familiar with the detailed requirements.

Principle: A tester picking up any single scenario row should be able to understand what they are verifying, execute the steps, and evaluate the result — without reading anything else in the document except the section Pre-conditions.

Clarity rules

Applying clarity

For each scenario row:

1. Read the Scenario, Steps, and Expected Result columns
2. Ask: "If I handed just this row (plus the pre-conditions block) to someone who has never seen this feature, could they test it?"
3. If no — expand the description, spell out the steps, or simplify the language
4. Check section titles, Known Limitations, and Pre-conditions against rules C5–C7

Scenario independence test

For any scenario that references another scenario, a prior state, or uses shorthand:

• Replace the reference with the actual content
• Replace shorthand ("Trigger a notification") with explicit steps ("Disable location permissions during a geo-required shift to trigger a notification")
• Exception: Pre-conditions are shared per section and do not need to be repeated in every row

Phase 5: Output

Save the file

Save to: {workspace}/Github-Issue-Extractor/issues/reports/uat-{descriptor}-{today}.md

Where {descriptor} is:

• If 1 feature group: slugified group name (e.g. uat-push-notifications-2026-03-24.md)
• If 2-3 groups: joined slug (e.g. uat-push-notifications-demo-experience-2026-03-24.md)
• If 4+ groups: uat-release-{today}.md

Chat summary

After saving, present to the user:

## UAT Plan Generated

**File:** [path]
**Feature groups:** [N]
**Total scenarios:** [N]
**Platforms:** iOS [n], Android [n], Backend [n]

| Feature Group | Scenarios | Platforms |
|---------------|-----------|-----------|
| [name]        | [n]       | iOS, Android |

**Purity rewrites:** [N] scenarios cleaned up
**Deferred items:** [N] (see Appendix A)
**Tickets without ACs:** [N] (see Appendix B)
**Technical verification:** [N] backend-only scenarios (see Appendix C)

Integration with Other Skills

Critical Rules

1. ACs are the primary source of truth. Every UAT scenario must trace back to a specific AC or, for defect tickets, to documented reproduction steps / expected behavior. Do not invent scenarios that are not grounded in ticket content.
2. Comments supplement, never override. Comments provide test steps and scope notes. If a comment contradicts an AC, flag it in Known Limitations — do not silently change the scenario.
3. Never fabricate test steps. If an AC has no Given/When/Then and no test steps in comments, use the AC description as the scenario and write "See scenario description" in the Steps column. For defects, derive steps from reproduction steps or title — do not invent ungrounded steps.
4. Purity is mandatory. Every scenario that stays in the main table must pass all 7 purity rules. Move impure scenarios to Technical Verification rather than leaving them in.
5. Clarity is mandatory. Every scenario must pass all 7 clarity rules (Phase 4.5). A first-time tester must be able to pick up any single row, read the scenario description, execute the steps, and evaluate the result — without reading any other scenario in the document.
6. Deferred items must be visible. If a comment says something is deferred or limited, it must appear in the Known Limitations and Issues table (with Type = Deferred or On Hold). Do not hide scope gaps.
7. Ask before generating. Always confirm the filtered ticket list with the user before reading issue files and generating the document. The confirmation step in Phase 1 Step 5 is not optional.
8. Defects belong in the scenario table, not the appendix. Most defect tickets are client-testable bug fixes. Use Phase 2f to extract reproduction steps and generate regression verification scenarios. Only park a ticket in "Appendix A: Tickets Without Testable Content" if it truly has no ACs, no repro steps, and no testable description.