Forge

Forge — Autonomous Quality Engineering Swarm

Quality forged in, not bolted on.

Forge is a self-learning, autonomous quality engineering swarm that unifies three approaches into one:

"DONE DONE" means: the code compiles AND the product behaves as specified. Every Gherkin scenario passes. Every quality gate clears. Every dependency graph is satisfied.

ARCHITECTURE ADAPTABILITY

Forge adapts to any project architecture. Before first run, it discovers your project structure:

Supported Architectures

Project Discovery

On first invocation, Forge analyzes the project to build a context map:

# Forge automatically discovers:
# 1. Backend technology (Rust/Cargo, Node/npm, Python/pip, Go, Java/Maven/Gradle, .NET)
# 2. Frontend technology (Flutter, React, Next.js, Vue, Angular, SwiftUI, Kotlin/Compose)
# 3. Test framework (integration_test, Jest, Pytest, Go test, JUnit, xUnit)
# 4. Project structure (monorepo layout, service boundaries, module boundaries)
# 5. API protocol (REST, GraphQL, gRPC, WebSocket)
# 6. Build system (Make, npm scripts, Gradle tasks, Cargo features)

Forge stores the discovered project map:

{
  "architecture": "mobile-backend",
  "backend": {
    "technology": "rust",
    "buildCommand": "cargo build --release --features test-endpoints",
    "runCommand": "cargo run --release --features test-endpoints",
    "healthEndpoint": "/health",
    "port": 8080,
    "migrationCommand": "cargo sqlx migrate run"
  },
  "frontend": {
    "technology": "flutter",
    "testCommand": "flutter drive --driver=test_driver/integration_test.dart --target={target}",
    "testDir": "integration_test/e2e/",
    "specDir": "integration_test/e2e/specs/"
  },
  "contexts": ["identity", "rides", "payments", "..."],
  "testDataSeeding": {
    "method": "api",
    "endpoint": "/api/v1/test/seed",
    "authHeader": "X-Test-Key"
  }
}

Configuration Override

Projects can provide a forge.config.yaml at the repo root to override auto-discovery:

# forge.config.yaml (optional — Forge auto-discovers if absent)
architecture: microservices
backend:
  services:
    - name: auth-service
      port: 8081
      healthEndpoint: /health
      buildCommand: npm run build
      runCommand: npm start
    - name: payment-service
      port: 8082
      healthEndpoint: /health
      buildCommand: npm run build
      runCommand: npm start
frontend:
  technology: react
  testCommand: npx cypress run --spec {target}
  testDir: cypress/e2e/
  specDir: cypress/e2e/specs/
contexts:
  - name: identity
    testFile: auth.cy.ts
    specFile: identity.feature
  - name: payments
    testFile: payments.cy.ts
    specFile: payments.feature
dependencies:
  identity:
    blocks: [payments, orders]
  payments:
    depends_on: [identity]
    blocks: [orders]

CRITICAL: NO MOCKING OR STUBBING ALLOWED

ABSOLUTE RULE: This skill NEVER uses mocking or stubbing of the backend API.

• ALL tests run against the REAL backend API
• NO mocking frameworks for API calls (no mockito, wiremock, MockClient, nock, msw, httpretty, etc.)
• NO stubbed responses or fake data from API endpoints
• The backend MUST be running and healthy before any tests execute
• Test data is seeded through REAL API calls, not mocked state

Why No Mocking:

• Mocks hide real integration bugs
• Mocks create false confidence
• Mocks don't test the actual data flow
• Real API tests catch serialization, validation, and timing issues

PHASE 0: BACKEND SETUP (MANDATORY FIRST STEP)

BEFORE ANY TESTING, the backend MUST be built, compiled, and running.

This is the FIRST thing the skill does — no exceptions.

Step 1: Check and Start Backend

# 1. Read project config or auto-discover backend settings
# 2. Check if backend is already running
curl -s http://localhost:${BACKEND_PORT}/${HEALTH_ENDPOINT} || {
  echo "Backend not running. Starting..."

  # 3. Navigate to backend directory
  cd ${BACKEND_DIR}

  # 4. Ensure environment is configured
  cp .env.example .env 2>/dev/null || true

  # 5. Build the backend
  ${BUILD_COMMAND}

  # 6. Run database migrations (if applicable)
  ${MIGRATION_COMMAND}

  # 7. Start backend (background)
  nohup ${RUN_COMMAND} > backend.log 2>&1 &
  echo $! > backend.pid

  # 8. Wait for backend to be healthy (up to 60 seconds)
  for i in {1..60}; do
    if curl -s http://localhost:${BACKEND_PORT}/${HEALTH_ENDPOINT} | grep -q "ok\|healthy\|UP"; then
      echo "Backend healthy on port ${BACKEND_PORT}"
      break
    fi
    sleep 1
  done
}

Step 2: Verify Backend Health

# Verify critical endpoints are responding
curl -s http://localhost:${BACKEND_PORT}/${HEALTH_ENDPOINT} | jq .

# Verify test fixtures endpoint (for seeding)
curl -s -H "${TEST_AUTH_HEADER}" http://localhost:${BACKEND_PORT}/${TEST_STATUS_ENDPOINT} | jq .

Step 3: Contract Validation

# Verify API spec matches running API (if OpenAPI/Swagger available)
curl -s http://localhost:${BACKEND_PORT}/${OPENAPI_ENDPOINT} > /tmp/live-spec.json

# Store contract snapshot for regression detection
npx @claude-flow/cli@latest memory store \
  --key "contract-snapshot-$(date +%s)" \
  --value "$(cat /tmp/live-spec.json | head -c 5000)" \
  --namespace forge-contracts

Step 4: Seed Test Data (Real API Calls)

# Seed test data through REAL API — adapt to your project's seeding endpoint
curl -X POST http://localhost:${BACKEND_PORT}/${SEED_ENDPOINT} \
  -H "Content-Type: application/json" \
  -H "${TEST_AUTH_HEADER}" \
  -d '${SEED_PAYLOAD}'

PHASE 1: BEHAVIORAL SPECIFICATION & ARCHITECTURE RECORDS

Before testing, verify Gherkin specs and architecture decision records exist for the target bounded context.

Behavioral specifications define WHAT the product does from the user's perspective. Every test traces back to a Gherkin scenario. If tests pass but specs fail, the product is broken.

Spec Location

Gherkin specs are stored alongside tests:

${SPEC_DIR}/
├── [context-a].feature
├── [context-b].feature
├── [context-c].feature
└── ...

The exact location depends on your project's test structure. Forge auto-discovers this from the project map.

Spec-to-Test Mapping

Each Gherkin Scenario maps to exactly one test function. The mapping is tracked:

Feature: [Context Name]
  As a [user role]
  I want to [action]
  So that [outcome]

  Scenario: [Descriptive scenario name]
    Given [precondition]
    When [action]
    Then [expected result]
    And [additional verification]

Missing Spec Generation

If specs are missing for a target context, the Specification Verifier agent creates them:

1. Read the screen/component/route implementation files for the context
2. Extract all user-visible features, interactions, and states
3. Generate Gherkin scenarios covering every cyclomatic path
4. Write to ${SPEC_DIR}/[context].feature
5. Map each scenario to its corresponding test function

Agent-Optimized ADR Generation

When Forge discovers a bounded context without an Architecture Decision Record, the Specification Verifier generates one. ADRs follow an agent-optimized format designed for machine consumption:

# ADR-NNN: [Context] Architecture Decision

## Status
Proposed | Accepted | Deprecated | Superseded by ADR-XXX

## MUST
- [Explicit required behaviors with contract references]
- [Link to OpenAPI spec: /api/v1/[context]/openapi.json]
- [Required integration patterns]

## MUST NOT
- [Explicit forbidden patterns]
- [Anti-patterns to avoid]
- [Coupling violations]

## Verification
- Command: [command to verify this decision holds]
- Expected: [expected output or exit code]

## Dependencies
- Depends on: [list of upstream contexts with ADR links]
- Blocks: [list of downstream contexts with ADR links]

ADR Storage:

• ADRs are stored in docs/decisions/ or the project-configured ADR directory
• Each bounded context has exactly one ADR
• ADRs are updated when contracts change or new dependencies are discovered
• The Specification Verifier agent includes ADR generation in its workflow

PHASE 2: CONTRACT & DEPENDENCY VALIDATION

Contract Validation

Before running tests, verify API response schemas match expected DTOs:

# For each endpoint the context uses:
# 1. Make a real API call
# 2. Compare response structure against expected DTO/schema
# 3. Flag any mismatches as contract violations

Contract violations are treated as Gate 7 failures and must be resolved before functional testing proceeds.

Shared Types Validation

For bounded contexts that share dependencies, validate type consistency across context boundaries:

1. Identify shared DTOs/models — For each context, extract types used in API requests and responses
2. Cross-reference types — Compare DTOs between contexts that share dependencies (from the dependency graph)
3. Flag type mismatches — e.g., context A expects userId: string but context B sends userId: number
4. Validate value objects — Ensure value objects (email, money, address) follow consistent patterns across contexts
5. Report violations — Flag as pre-Gate warnings with specific file locations and expected vs actual types

{
  "sharedTypeViolation": {
    "type": "UserId",
    "contextA": { "name": "payments", "file": "types/payment.ts", "definition": "string" },
    "contextB": { "name": "orders", "file": "types/order.ts", "definition": "number" },
    "severity": "error"
  }
}

Cross-Cutting Foundation Validation

Verify cross-cutting concerns are consistent across all bounded contexts:

• Auth patterns — Same header format (Authorization: Bearer <token>), same token validation approach across all endpoints
• Error response format — All API endpoints return errors in the project's standard format (consistent structure, error codes, HTTP status codes)
• Logging patterns — Consistent log levels, structured format, and correlation IDs across contexts
• Pagination format — Consistent pagination parameters and response format across collection endpoints

Cross-cutting violations are reported as warnings before Gate evaluation begins.

Dependency Graph

Bounded contexts have dependencies. When a fix touches context X, all contexts that depend on X must be re-tested.

# Context Dependency Map — define in forge.config.yaml or auto-discover
# Example for a typical application:
#
# authentication:
#   depends_on: []
#   blocks: [orders, payments, profile, messaging]
#
# payments:
#   depends_on: [authentication]
#   blocks: [orders, subscriptions]
#
# orders:
#   depends_on: [authentication, payments]
#   blocks: [reviews, notifications]

Cascade Re-Testing

When Bug Fixer modifies a file in context X:

1. Identify which context X belongs to
2. Look up all contexts in blocks list for X
3. After X's tests pass, automatically re-run tests for blocked contexts
4. If a cascade failure occurs, trace it back to the original fix

PHASE 3: SWARM INITIALIZATION

# Initialize anti-drift swarm for Forge
npx @claude-flow/cli@latest swarm init --topology hierarchical --max-agents 10 --strategy specialized

# Load previous fix patterns from memory
npx @claude-flow/cli@latest memory search --query "forge fix patterns" --namespace forge-patterns

# Check current coverage and gate status
npx @claude-flow/cli@latest memory retrieve --key "forge-coverage-status" --namespace forge-state

# Load confidence tiers
npx @claude-flow/cli@latest memory search --query "confidence tier" --namespace forge-patterns

# Check defect predictions for target context
npx @claude-flow/cli@latest memory search --query "defect prediction" --namespace forge-predictions

MODEL ROUTING

Forge routes each agent to the appropriate model tier based on task complexity, optimizing for cost without sacrificing quality:

Projects can override model assignments in forge.config.yaml:

# forge.config.yaml — Model routing overrides (optional)
model_routing:
  spec-verifier: sonnet
  test-runner: haiku
  failure-analyzer: sonnet
  bug-fixer: opus
  gate-enforcer: haiku
  accessibility-auditor: sonnet
  auto-committer: haiku
  learning-optimizer: sonnet

When no override is specified, the defaults above are used. This routing reduces token cost by ~60% compared to running all agents on the highest-tier model.

PHASE 4: SPAWN AUTONOMOUS AGENTS

Claude Code MUST spawn these 8 agents in a SINGLE message with run_in_background: true:

// Agent 1: Specification Verifier
Task({
  model: "sonnet",
  prompt: `You are the Specification Verifier agent. Your mission:

    1. VERIFY backend is running: curl -sf http://localhost:${BACKEND_PORT}/${HEALTH_ENDPOINT}
    2. Check if Gherkin specs exist for the target bounded context:
       - Look in the project's spec directory
    3. If specs are MISSING:
       - Read the screen/component/route implementation files for the context
       - Extract all user-visible features, interactions, states
       - Generate Gherkin feature files with scenarios for every cyclomatic path
       - Write specs to the correct location
    4. If specs EXIST:
       - Read current implementations
       - Compare against existing scenarios
       - Flag scenarios that no longer match implementation (stale specs)
       - Generate new scenarios for uncovered features
    5. Create spec-to-test mapping:
       - Each Scenario name → test function name
       - Store mapping in memory for Test Runner
    6. Store results:
       npx @claude-flow/cli@latest memory store --key "specs-[context]-[timestamp]" \
         --value "[spec status JSON]" --namespace forge-specs

    CONSTRAINTS:
    - NEVER generate specs for code you haven't read
    - NEVER assume UI elements exist without checking implementation
    - NEVER create scenarios that duplicate existing coverage
    - NEVER modify existing test files — only spec files

    ACCEPTANCE:
    - Every implementation file has at least one Gherkin scenario
    - Spec-to-test mapping has zero unmapped entries
    - All generated scenarios follow Given/When/Then format
    - Results stored in forge-specs namespace

    Output: List of all Gherkin scenarios with their mapped test functions, and any gaps found.`,
  subagent_type: "researcher",
  description: "Spec Verification",
  run_in_background: true
})

// Agent 2: Test Runner
Task({
  model: "haiku",
  prompt: `You are the Test Runner agent. Your mission:

    1. VERIFY backend is running
    2. Check defect predictions from memory:
       npx @claude-flow/cli@latest memory search --query "defect prediction [context]" --namespace forge-predictions
       - Run predicted-to-fail tests FIRST for faster convergence
    3. Run the E2E test suite for the specified context using the project's test command
    4. Capture ALL test output including stack traces
    5. Parse failures into structured format:
       {testId, gherkinScenario, error, stackTrace, file, line, context}
    6. Map each failure to its Gherkin scenario (from spec-to-test mapping)
    7. Store results in memory for other agents:
       npx @claude-flow/cli@latest memory store \
         --key "test-run-[timestamp]" \
         --value "[parsed results JSON]" \
         --namespace forge-results

    CONSTRAINTS:
    - NEVER skip failing tests
    - NEVER modify test code or source code
    - NEVER mock API calls or stub responses
    - NEVER continue if backend health check fails

    ACCEPTANCE:
    - All test results stored in memory with structured format
    - Zero unparsed failures — every failure has testId, error, stackTrace, file, line
    - Predicted-to-fail tests executed first
    - Results include Gherkin scenario mapping for every test`,
  subagent_type: "tester",
  description: "Test Runner",
  run_in_background: true
})

// Agent 3: Failure Analyzer
Task({
  model: "sonnet",
  prompt: `You are the Failure Analyzer agent. Your mission:

    1. Monitor memory for new test results from Test Runner
    2. For each failure, analyze:
       - Root cause category: element-not-found, assertion-failed, timeout,
         api-mismatch, navigation-error, state-error, contract-violation
       - Affected file and line number
       - Which Gherkin scenario is violated
       - Impact on dependent contexts (check dependency graph)
    3. Search memory for matching fix patterns with confidence tiers:
       npx @claude-flow/cli@latest memory search \
         --query "[error pattern]" --namespace forge-patterns
    4. If pattern found with confidence >= 0.85 (Gold+):
       - Recommend auto-apply
       - Include pattern key and success rate
    5. If pattern found with confidence >= 0.75 (Silver):
       - Suggest fix but flag for review
    6. If no matching pattern:
       - Perform root cause analysis from first principles
       - Generate fix hypothesis
    7. Store analysis in memory for Bug Fixer:
       npx @claude-flow/cli@latest memory store \
         --key "analysis-[testId]-[timestamp]" \
         --value "[analysis JSON]" \
         --namespace forge-results

    CONSTRAINTS:
    - NEVER assume root cause without stack trace evidence
    - NEVER recommend fixes for passing tests
    - NEVER skip dependency graph impact analysis
    - NEVER override confidence tier thresholds

    ACCEPTANCE:
    - Every failure has a root cause category and affected file
    - Zero unanalyzed failures
    - Dependency impact documented for every failure
    - Pattern search executed for every error type`,
  subagent_type: "researcher",
  description: "Failure Analyzer",
  run_in_background: true
})

// Agent 4: Bug Fixer
Task({
  model: "opus",
  prompt: `You are the Bug Fixer agent. Your mission:

    1. Retrieve failure analysis from memory
    2. For each failure, apply fix using confidence-tiered approach:

       PLATINUM (>= 0.95 confidence):
       - Auto-apply the stored fix pattern immediately
       - No review needed

       GOLD (>= 0.85 confidence):
       - Auto-apply the stored fix pattern
       - Flag in commit message for awareness

       SILVER (>= 0.75 confidence):
       - Read the failing test file and source file
       - Apply suggested fix with extra verification
       - Run targeted test before proceeding

       BRONZE or NO PATTERN:
       - Read the failing test file
       - Read the source file causing the failure
       - Implement fix from first principles
       - Use defensive patterns appropriate to the test framework

    3. After fixing, identify affected context:
       - Check dependency graph for cascade impacts
       - Flag dependent contexts for re-testing

    4. Store the fix pattern with initial confidence:
       npx @claude-flow/cli@latest memory store \
         --key "fix-[error-type]-[hash]" \
         --value '{"pattern":"[fix]","confidence":0.75,"tier":"silver","applied":1,"successes":0}' \
         --namespace forge-patterns

    5. Signal Test Runner to re-run affected tests
    6. Signal Quality Gate Enforcer to check all 7 gates

    CONSTRAINTS:
    - NEVER change test assertions to make tests pass
    - NEVER modify Gherkin specs to match broken behavior
    - NEVER introduce new dependencies without flagging
    - NEVER apply fixes without reading both test file and source file

    ACCEPTANCE:
    - Every applied fix has a targeted test re-run result
    - Zero fixes without verification
    - Fix pattern stored with initial confidence score
    - Cascade impacts identified and flagged for re-testing`,
  subagent_type: "coder",
  description: "Bug Fixer",
  run_in_background: true
})

// Agent 5: Quality Gate Enforcer
Task({
  model: "haiku",
  prompt: `You are the Quality Gate Enforcer agent. Your mission:

    After each fix cycle, evaluate ALL 7 quality gates:

    GATE 1 — FUNCTIONAL (100% required):
    - All tests in the target context pass
    - No regressions in previously passing tests

    GATE 2 — BEHAVIORAL (100% of targeted scenarios):
    - Every Gherkin scenario that was targeted has a passing test
    - Spec-to-test mapping is complete (no unmapped scenarios)

    GATE 3 — COVERAGE (>=85% overall, >=95% critical paths):
    - Calculate path coverage for the context
    - Critical paths: authentication, payment, core workflows
    - Non-critical paths: preferences, history, settings

    GATE 4 — SECURITY (0 critical/high violations):
    - No hardcoded API keys, tokens, or secrets in test files
    - No hardcoded test credentials (use env vars or test fixtures)
    - Secure storage patterns used (no plaintext sensitive data)
    - No SQL injection vectors in dynamic queries
    - No XSS vectors in rendered output
    - No path traversal in file operations
    - Dependencies have no known critical CVEs (when lockfile available)
    - When AQE available: delegate to security-scanner for full SAST analysis

    GATE 5 — ACCESSIBILITY (WCAG AA):
    - All interactive elements have accessible labels
    - Touch/click targets meet minimum size requirements
    - Color contrast meets WCAG AA ratios
    - Screen reader navigation order is logical

    GATE 6 — RESILIENCE (tested for target context):
    - Offline/disconnected state handled gracefully
    - Timeout handling shows user-friendly message
    - Error states show retry option
    - Server errors show generic error, not stack trace

    GATE 7 — CONTRACT (0 mismatches):
    - API responses match expected schemas
    - No unexpected null fields
    - Enum values match expected set
    - Pagination format is consistent

    For each gate:
    - Status: PASS / FAIL / SKIP (with reason)
    - Details: what passed, what failed
    - Blocking: whether this gate blocks the commit

    Store gate results:
    npx @claude-flow/cli@latest memory store \
      --key "gates-[context]-[timestamp]" \
      --value "[gate results JSON]" \
      --namespace forge-state

    ONLY signal Auto-Committer when ALL 7 GATES PASS.

    CONSTRAINTS:
    - NEVER approve a commit with ANY blocking gate failure
    - NEVER lower thresholds below defined minimums
    - NEVER skip gate evaluation — all 7 gates must be assessed
    - NEVER mark a gate as PASS without evidence

    ACCEPTANCE:
    - Gate results stored in memory with PASS/FAIL/SKIP for all 7 gates
    - Every FAIL includes specific details of what failed
    - Every SKIP includes reason for skipping
    - Auto-Committer only signaled when all blocking gates pass`,
  subagent_type: "reviewer",
  description: "Quality Gate Enforcer",
  run_in_background: true
})

// Agent 6: Accessibility Auditor
Task({
  model: "sonnet",
  prompt: `You are the Accessibility Auditor agent. Your mission:

    1. For each screen/page/component in the target context, audit:

    LABELS:
    - Every interactive element has an accessible label/aria-label/Semantics label
    - Labels are descriptive (not "button1" but "Submit payment")
    - Images have alt text or semantic labels

    TOUCH/CLICK TARGETS:
    - All interactive elements meet minimum size (48x48dp mobile, 44x44px web)
    - Flag any undersized targets

    CONTRAST:
    - Text on colored backgrounds meets WCAG AA ratio (4.5:1 normal, 3:1 large)
    - Flag low-contrast combinations

    SCREEN READER:
    - Accessibility tree has logical reading order
    - No duplicate or misleading labels
    - Form fields have associated labels

    FOCUS/TAB ORDER:
    - Focus order follows visual layout
    - Focus trap in modals/dialogs
    - Focus returns to trigger after dialog closes

    2. Generate findings as:
       {severity: "critical"|"warning"|"info", element, file, line, issue, fix}

    3. Store audit results:
       npx @claude-flow/cli@latest memory store \
         --key "a11y-[context]-[timestamp]" \
         --value "[audit JSON]" \
         --namespace forge-state

    CONSTRAINTS:
    - NEVER skip interactive elements during audit
    - NEVER report false positives for decorative images
    - NEVER ignore focus/tab order analysis
    - NEVER apply fixes — only report findings for Bug Fixer

    ACCEPTANCE:
    - Every interactive element audited
    - Findings stored with severity, element, file, line, issue, fix
    - Zero unaudited interactive elements in target context
    - WCAG AA compliance level assessed for every screen`,
  subagent_type: "analyst",
  description: "Accessibility Auditor",
  run_in_background: true
})

// Agent 7: Auto-Committer
Task({
  model: "haiku",
  prompt: `You are the Auto-Committer agent. Your mission:

    1. Monitor for successful fixes where ALL 7 QUALITY GATES PASS
    2. For each successful fix:
       - Stage only the fixed files (never git add -A)
       - Create detailed commit message:

         fix(forge): Fix [TEST_ID] - [brief description]

         Behavioral Spec: [Gherkin scenario name]
         Root Cause: [what caused the failure]
         - [specific issue 1]
         - [specific issue 2]

         Fix Applied:
         - [change 1]
         - [change 2]

         Quality Gates:
         - Functional: PASS
         - Behavioral: PASS
         - Coverage: [X]%
         - Security: PASS
         - Accessibility: PASS
         - Resilience: PASS
         - Contract: PASS

         Confidence Tier: [platinum|gold|silver|bronze]
         Pattern Stored: fix-[error-type]-[hash]

       - Commit with the message above
    3. Update coverage report with new passing paths
    4. Store commit hash in memory for rollback capability:
       npx @claude-flow/cli@latest memory store \
         --key "commit-[hash]" \
         --value "[commit details JSON]" \
         --namespace forge-commits
    5. Store last known good commit:
       npx @claude-flow/cli@latest memory store \
         --key "last-green-commit" \
         --value "[hash]" \
         --namespace forge-state

    CONSTRAINTS:
    - NEVER use git add -A or git add .
    - NEVER commit without all 7 gates passing
    - NEVER amend previous commits
    - NEVER push to remote — only local commits

    ACCEPTANCE:
    - Commit message includes Behavioral Spec, Root Cause, Fix Applied, all 7 gate statuses
    - Only fixed files are staged (no unrelated files)
    - Commit hash stored in forge-commits namespace
    - Last green commit updated in forge-state namespace`,
  subagent_type: "reviewer",
  description: "Auto-Committer",
  run_in_background: true
})

// Agent 8: Learning Optimizer
Task({
  model: "sonnet",
  prompt: `You are the Learning Optimizer agent. Your mission:

    1. After each test cycle, analyze patterns:
       - Which error types fail most often?
       - Which fix patterns have highest success rate?
       - What new defensive patterns should be added?
       - Which Gherkin scenarios are most fragile?

    2. UPDATE CONFIDENCE TIERS:
       For each fix pattern applied this cycle:
       - If fix succeeded: confidence += 0.05 (cap at 1.0)
         - If confidence crosses 0.95: promote to Platinum
         - If confidence crosses 0.85: promote to Gold
       - If fix failed: confidence -= 0.10 (floor at 0.0)
         - If confidence drops below 0.70: demote to Bronze (learning-only)
       Store updated pattern:
       npx @claude-flow/cli@latest memory store \
         --key "fix-[error-type]-[hash]" \
         --value "[updated pattern JSON]" \
         --namespace forge-patterns

    3. DEFECT PREDICTION:
       Analyze which contexts/files are likely to fail next:
       - Files changed since last green run
       - Historical failure rate per context
       - Complexity of recent changes
       Store prediction:
       npx @claude-flow/cli@latest memory store \
         --key "prediction-[date]" \
         --value "[prediction JSON]" \
         --namespace forge-predictions

    4. Train neural patterns on successful fixes:
       npx @claude-flow/cli@latest hooks post-task \
         --task-id "forge-cycle" --success true --store-results true

    5. Update coverage status:
       npx @claude-flow/cli@latest memory store \
         --key "forge-coverage-status" \
         --value "[updated coverage JSON]" \
         --namespace forge-state

    6. Generate recommendations for test improvements
    7. Export learning metrics:
       npx @claude-flow/cli@latest neural train --pattern-type forge-fixes --epochs 5

    CONSTRAINTS:
    - NEVER promote a pattern that failed in the current cycle
    - NEVER delete patterns — only demote below Bronze threshold
    - NEVER override confidence scores without evidence from test results
    - NEVER generate predictions without historical data

    ACCEPTANCE:
    - All applied patterns have updated confidence scores
    - Prediction stored for next run with context-level probabilities
    - Coverage status updated in forge-state namespace
    - Zero patterns promoted without success evidence`,
  subagent_type: "researcher",
  description: "Learning Optimizer",
  run_in_background: true
})

PHASE 5: QUALITY GATES

7 gates evaluated after each fix cycle. ALL must pass before a commit is created.

Gate Failure Categories

When gates fail, failures are categorized for targeted re-runs:

• Functional failures → Re-run Bug Fixer on failing tests
• Behavioral failures → Check spec-to-test mapping, may need new tests
• Coverage failures → Generate additional test paths
• Security failures → Fix hardcoded values, update storage patterns
• Accessibility failures → Add accessible labels, fix target sizes
• Resilience failures → Add offline/error state handling
• Contract failures → Update DTOs or flag API regression

AUTONOMOUS EXECUTION LOOP

┌────────────────────────────────────────────────────────────────────────┐
│                      FORGE AUTONOMOUS LOOP                             │
├────────────────────────────────────────────────────────────────────────┤
│                                                                        │
│  ┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────┐        │
│  │ Specify  │───▶│   Test   │───▶│ Analyze  │───▶│   Fix    │        │
│  │ (Gherkin)│    │ (Run)    │    │ (Root    │    │ (Tiered) │        │
│  └──────────┘    └──────────┘    │  Cause)  │    └──────────┘        │
│       ▲                          └──────────┘         │               │
│       │                                               ▼               │
│  ┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────┐        │
│  │  Learn   │◀───│  Commit  │◀───│  Gate    │◀───│  Audit   │        │
│  │ (Update  │    │ (Auto)   │    │ (7 Gates)│    │ (A11y)   │        │
│  │  Tiers)  │    └──────────┘    └──────────┘    └──────────┘        │
│  └──────────┘                                                         │
│       │                                                                │
│       └───────────────── REPEAT ──────────────────────────────────────│
│                                                                        │
│  Loop continues until: ALL 7 GATES PASS or MAX_ITERATIONS (10)        │
│  Gate failures are categorized for targeted re-runs (not full re-run) │
└────────────────────────────────────────────────────────────────────────┘

REAL-TIME PROGRESS REPORTING

Each agent emits structured progress events during execution for observability:

{"agent": "spec-verifier", "event": "spec_generated", "context": "payments", "scenarios": 12}
{"agent": "test-runner", "event": "test_started", "context": "payments", "test": "user_can_pay"}
{"agent": "test-runner", "event": "test_completed", "context": "payments", "passed": 10, "failed": 2}
{"agent": "failure-analyzer", "event": "root_cause_found", "test": "user_can_pay", "cause": "timeout"}
{"agent": "bug-fixer", "event": "fix_applied", "file": "payments.ts", "confidence": 0.92}
{"agent": "gate-enforcer", "event": "gate_evaluated", "gate": "functional", "status": "PASS"}
{"agent": "auto-committer", "event": "committed", "hash": "abc123", "tests_fixed": 2}
{"agent": "learning-optimizer", "event": "pattern_updated", "pattern": "fix-timeout-xyz", "tier": "gold"}

Progress File:

• Events are appended to .forge/progress.jsonl (one JSON object per line)
• File is created at the start of each Forge run and truncated
• Tools can tail this file for real-time monitoring: tail -f .forge/progress.jsonl

Integration with Agentic QE AG-UI:

• When the AQE AG-UI protocol is available, events stream directly to the user interface
• Users see live progress: which gate is being evaluated, which test is running, which fix is being applied
• When running in Claude Code without AG-UI, progress is visible through agent output files

CONFIDENCE TIERS FOR FIX PATTERNS

Every fix pattern is tracked with a confidence score that evolves over time:

{
  "key": "fix-element-not-found-abc123",
  "pattern": {
    "error": "Element not found / No element",
    "fix": "Ensure element is rendered and visible before interaction",
    "files_affected": ["*_test.*"],
    "context": "any"
  },
  "tier": "gold",
  "confidence": 0.92,
  "auto_apply": true,
  "applied_count": 47,
  "success_count": 43,
  "success_rate": 0.915,
  "last_applied": "2026-02-06T14:30:00Z",
  "last_failed": "2026-02-01T09:15:00Z"
}

Tier Thresholds

Confidence Updates

After each application:

• Success: confidence += 0.05 (capped at 1.0)
• Failure: confidence -= 0.10 (floored at 0.0)
• Tier promotion when crossing threshold upward
• Tier demotion when crossing threshold downward

DEFECT PREDICTION

Before running tests, the Learning Optimizer analyzes historical data to predict which tests are most likely to fail:

Input Signals

1. Files changed since last green run (git diff against last-green-commit)
2. Historical failure rates per bounded context (from forge-results namespace)
3. Fix pattern freshness — recently applied fixes are more likely to regress
4. Complexity metrics — contexts with more cyclomatic paths fail more often
5. Dependency chain length — deeper dependency chains have higher failure rates

Prediction Output

{
  "date": "2026-02-07",
  "predictions": [
    { "context": "payments", "probability": 0.73, "reason": "3 files changed in payment module" },
    { "context": "orders", "probability": 0.45, "reason": "depends on payments (changed)" },
    { "context": "identity", "probability": 0.12, "reason": "no changes, stable history" }
  ],
  "recommended_order": ["payments", "orders", "identity"]
}

Tests are executed in descending probability order — predicted-to-fail tests run FIRST for faster convergence.

EXHAUSTIVE EDGE CASE TESTING

General UI Element Edge Cases

For EVERY interactive element, test:

1. Interaction States

   • Single interaction → expected action
   • Repeated rapid interaction → no duplicate action
   • Long press / right-click → context menu if applicable
   • Disabled state → no action, visual feedback

2. Input Field States

   • Empty → placeholder visible
   • Focus → visual focus indicator
   • Valid input → no error
   • Invalid input → error message
   • Max length reached → prevents further input
   • Paste → validates pasted content
   • Clear → resets to empty

3. Async Operation States

   • Before load → loading indicator
   • During load → spinner, disabled submit
   • Success → data displayed, spinner gone
   • Error → error message, retry option
   • Timeout → timeout message, retry option

4. Navigation Edge Cases

   • Back navigation → previous screen or exit confirmation
   • Deep link → correct screen with params
   • Invalid deep link → fallback/error screen
   • Browser forward/back (web) → correct state

5. Scroll Edge Cases

   • Overscroll → appropriate feedback
   • Scroll to hidden content → content becomes visible
   • Keyboard appears → scroll to focused field

Network Edge Cases

1. No internet → offline indicator, cached data if available
2. Slow connection → loading states persist, timeout handling
3. Connection restored → auto-retry pending operations
4. Server error 500 → generic error message
5. Auth error 401 → redirect to login
6. Permission error 403 → permission denied message
7. Not found 404 → "not found" message

Chaos Testing (Resilience)

For each target context, inject controlled failures:

1. Timeout injection → API calls take >10s → verify timeout UI
2. Partial response → API returns incomplete data → verify graceful degradation
3. Rate limiting → API returns 429 → verify retry-after behavior
4. Concurrent mutations → Multiple clients modify same resource → verify conflict handling
5. Session expiry → Token expires mid-flow → verify re-auth prompt

Visual Regression

…(truncated)