# Start QA

> Execute QA tests from plan file

- Skill: `rlajous/start-qa` (Agent Skill)
- Install (CLI): `npx skillmds@latest add rlajous/start-qa`
- Raw SKILL.md: https://api.skillmd.com/api/skills/rlajous/start-qa/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: rlajous (https://skillmd.com/u/rlajous)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/rlajous/start-qa

---


> Cross-runtime: follow [runtime compatibility](../../references/runtime-compatibility.md) for invocation, delegation, configuration precedence, state paths, and permissions.

You are helping execute QA tests from a test plan file generated by `/plan-qa`. This command reads the YAML test plan, executes each test case, and reports results.

## Step 1: Load Configuration

Check for configuration:

```bash
if [ -f ".git-workflow/config.yaml" ]; then
  CONFIG_PATH=".git-workflow/config.yaml"
elif [ -f ".claude/config.yaml" ]; then
  CONFIG_PATH=".claude/config.yaml" # legacy read-only fallback
else
  CONFIG_PATH=""
fi
```

**Load from the resolved `CONFIG_PATH` (if one exists):**

```yaml
qa:
  apiBaseUrl: ${API_BASE_URL}
  testPlansDir: tests/qa
  resultsDir: tests/qa/results
  timeout: 10
```

**Default Values:**

```yaml
qa:
  testPlansDir: tests/qa
  resultsDir: tests/qa/results
  timeout: 10
```

## Step 2: Parse Arguments

Extract from `$ARGUMENTS`:

```
$ARGUMENTS
```

**Determine Test File:**

| Input                        | Resolution                                   |
| ---------------------------- | -------------------------------------------- |
| Direct path                  | Use as-is: `tests/qa/my-test.yaml`           |
| Ticket ID (PROJ-123)         | Look for: `{testPlansDir}/proj-123-test.yaml` |
| `latest`                     | Find newest `.yaml` in `{testPlansDir}`      |
| Empty                        | Find newest `.yaml` in `{testPlansDir}`      |

**Resolution Logic:**

```bash
TEST_DIR=$(config.qa.testPlansDir || "tests/qa")

if [ -f "$ARGUMENTS" ]; then
  # Direct path provided
  TEST_FILE="$ARGUMENTS"
elif [[ "$ARGUMENTS" =~ ^[A-Z]+-[0-9]+$ ]]; then
  # Ticket ID - look for matching file
  TICKET_LOWER=$(echo "$ARGUMENTS" | tr '[:upper:]' '[:lower:]')
  TEST_FILE="${TEST_DIR}/${TICKET_LOWER}-test.yaml"
elif [ "$ARGUMENTS" == "latest" ] || [ -z "$ARGUMENTS" ]; then
  # Find most recent test file
  TEST_FILE=$(ls -t ${TEST_DIR}/*.yaml 2>/dev/null | head -n 1)
fi
```

## Step 3: Load Test Plan

Read and parse the YAML test file:

```bash
if [ ! -f "$TEST_FILE" ]; then
  echo "Error: Test file not found: $TEST_FILE"
  exit 1
fi
```

**Extract Configuration:**

```yaml
name: "Test Plan Name"
base_url: "${API_BASE_URL}"
timeout: 10
test_cases: [...]
sqs_assertions: [...]
```

**Environment Variable Substitution:**

Replace `${VAR_NAME}` patterns with actual environment values.

## Step 4: Execute Test Cases

Immediately before delegating or executing the first test, summarize the plan file, environment/base URL, number and kinds of external operations, credentials or queues involved, and result destination. Ask for explicit confirmation to run the plan. Do not treat automatic skill invocation or plan-file selection as approval.

> **Optional — delegate execution.** For large or complex test plans (many endpoints, SQS event verification), delegate execution to the **`qa-executor`** agent through the active host subagent mechanism; it runs the plan and returns a detailed pass/fail report, keeping this session's context lean. For small plans, execute inline as below.

For each test case in the plan:

### HTTP Request Execution

Using curl or available tools:

```bash
# Build request
URL="${BASE_URL}${ENDPOINT}"
METHOD="${TEST_CASE.method}"
HEADERS="${TEST_CASE.headers}"
BODY="${TEST_CASE.body}"
PARAMS="${TEST_CASE.params}"

# Add query params
if [ -n "$PARAMS" ]; then
  URL="${URL}?$(urlencode $PARAMS)"
fi

# Execute request
START_TIME=$(date +%s%N)

RESPONSE=$(curl -s -w "\n%{http_code}" \
  -X "$METHOD" \
  -H "Content-Type: application/json" \
  ${HEADERS[@]} \
  -d "$BODY" \
  --max-time $TIMEOUT \
  "$URL")

END_TIME=$(date +%s%N)
DURATION_MS=$(( (END_TIME - START_TIME) / 1000000 ))

# Parse response
HTTP_CODE=$(echo "$RESPONSE" | tail -n 1)
RESPONSE_BODY=$(echo "$RESPONSE" | sed '$d')
```

### Using MCP Tools (If Available)

If MCP tools are available:

```
mcp__ai-qa-tools__check_api_access(
  base_url: BASE_URL,
  method: METHOD,
  path: ENDPOINT,
  headers: HEADERS,
  body: BODY,
  timeout: TIMEOUT
)
```

### Validate Response

**Status Code Check:**

```javascript
const passed = responseCode === expected.status;
```

**Body Assertions:**

```javascript
// Exact match
if (expected.body.field === response.body.field) {
  // Pass
}

// Wildcard match (*)
if (expected.body.id === '*' && response.body.id != null) {
  // Pass - any non-null value
}

// Array check
if (Array.isArray(expected.body.items) && Array.isArray(response.body.items)) {
  // Pass
}
```

### Record Result

```javascript
{
  id: "TC-001",
  name: "Test name",
  status: "PASS" | "FAIL" | "SKIP" | "ERROR",
  duration_ms: 150,
  expected_status: 200,
  actual_status: 200,
  assertions: [
    { field: "status", expected: 200, actual: 200, passed: true },
    { field: "body.success", expected: true, actual: true, passed: true }
  ],
  error: null | "Error message"
}
```

## Step 5: Execute SQS Assertions (If Defined)

For each SQS assertion:

### Using MCP Tools

```
mcp__ai-qa-tools__verify_sqs_message(
  queue_url: QUEUE_URL,
  match_mode: MATCH_MODE,
  expected_content: EXPECTED_CONTENT,
  expected_fields: EXPECTED_FIELDS,
  timeout_seconds: TIMEOUT
)
```

### Manual Verification

```bash
# Poll SQS queue
START_TIME=$(date +%s)
TIMEOUT_TIME=$((START_TIME + TIMEOUT_SECONDS))

while [ $(date +%s) -lt $TIMEOUT_TIME ]; do
  MESSAGES=$(aws sqs receive-message \
    --queue-url "$QUEUE_URL" \
    --max-number-of-messages 10 \
    --wait-time-seconds 5)

  # Check each message against expected fields
  for message in $MESSAGES; do
    if matches_expected "$message" "$EXPECTED_FIELDS"; then
      echo "SQS assertion passed"
      break 2
    fi
  done
done
```

## Step 6: Generate Results Report

### Console Output

```
═══════════════════════════════════════════════════════════════════════
  QA Test Results: {TEST_NAME}
═══════════════════════════════════════════════════════════════════════

Test Plan: {FILE_NAME}
Base URL: {BASE_URL}
Executed: {TIMESTAMP}

───────────────────────────────────────────────────────────────────────
  Test Cases: {PASSED}/{TOTAL} passed
───────────────────────────────────────────────────────────────────────

┌──────────┬────────────────────────────────────┬────────┬──────────┐
│ ID       │ Name                               │ Status │ Duration │
├──────────┼────────────────────────────────────┼────────┼──────────┤
│ TC-001   │ Happy path - get users             │ PASS   │ 145ms    │
│ TC-002   │ Invalid input returns 400          │ PASS   │ 52ms     │
│ TC-003   │ Missing auth returns 401           │ FAIL   │ 48ms     │
│ TC-004   │ Empty filter returns empty array   │ PASS   │ 67ms     │
└──────────┴────────────────────────────────────┴────────┴──────────┘

───────────────────────────────────────────────────────────────────────
  Failed Tests
───────────────────────────────────────────────────────────────────────

TC-003: Missing auth returns 401
  Expected: status 401
  Actual:   status 200
  Response: {"error": false, "data": [...]}

───────────────────────────────────────────────────────────────────────
  SQS Assertions: {PASSED}/{TOTAL} passed
───────────────────────────────────────────────────────────────────────

┌────────────────────────────┬────────┬───────────────┐
│ Assertion                  │ Status │ Found In      │
├────────────────────────────┼────────┼───────────────┤
│ USER_CREATED event         │ PASS   │ 2.3s          │
│ ORDER_PLACED event         │ FAIL   │ Timeout (30s) │
└────────────────────────────┴────────┴───────────────┘

═══════════════════════════════════════════════════════════════════════
  Overall: {PASS|FAIL} ({PASSED}/{TOTAL} tests passed)
═══════════════════════════════════════════════════════════════════════
```

### JSON Results File

Save detailed results:

```json
{
  "test_plan": "proj-123-test.yaml",
  "name": "Test Plan Name",
  "executed_at": "2025-01-17T12:00:00Z",
  "base_url": "http://localhost:3000",
  "summary": {
    "total": 4,
    "passed": 3,
    "failed": 1,
    "skipped": 0,
    "duration_ms": 312
  },
  "test_cases": [
    {
      "id": "TC-001",
      "name": "Happy path",
      "status": "PASS",
      "duration_ms": 145,
      "request": {
        "method": "GET",
        "url": "/api/v1/users",
        "headers": {}
      },
      "response": {
        "status": 200,
        "body": {}
      },
      "assertions": []
    }
  ],
  "sqs_assertions": [
    {
      "name": "USER_CREATED event",
      "status": "PASS",
      "found_in_seconds": 2.3
    }
  ]
}
```

## Step 7: Save Results

Before persisting results, redact authorization and cookie headers plus values whose keys contain `token`, `password`, `secret`, or `apiKey`. Save status, timing, validation differences, and sanitized excerpts by default. Capture full request or response bodies only after explicit approval for the current run, and redact them before writing.

```bash
RESULTS_DIR=$(config.qa.resultsDir || "tests/qa/results")
mkdir -p ${RESULTS_DIR}

TIMESTAMP=$(date +%Y%m%d-%H%M%S)
RESULTS_FILE="${RESULTS_DIR}/${TEST_NAME}-${TIMESTAMP}.json"

echo "$RESULTS_JSON" > "$RESULTS_FILE"
```

## Step 8: Confirm

```
Results saved: {RESULTS_FILE}

Summary:
- Test Cases: {PASSED}/{TOTAL} passed
- SQS Checks: {SQS_PASSED}/{SQS_TOTAL} passed
- Duration: {TOTAL_DURATION}ms

{If failed tests:}
Failed tests require attention:
- TC-003: Missing auth returns 401 (expected 401, got 200)
```

## Configuration Reference

| Setting           | Default             | Description                |
| ----------------- | ------------------- | -------------------------- |
| `qa.testPlansDir` | `tests/qa`          | Test plan directory        |
| `qa.resultsDir`   | `tests/qa/results`  | Results output directory   |
| `qa.timeout`      | `10`                | Default timeout (seconds)  |
| `qa.apiBaseUrl`   | -                   | Default API base URL       |

## Examples

### Run Latest Test

```
/start-qa
```

Runs the most recently modified test file.

### Run by Ticket ID

```
/start-qa PROJ-123
```

Runs `tests/qa/proj-123-test.yaml`.

### Run Specific File

```
/start-qa tests/qa/user-api-test.yaml
```

Runs the specified test file.

## Test File Format Reference

Expected YAML structure:

```yaml
name: "Test Name"
base_url: "${API_BASE_URL}"
timeout: 10

test_cases:
  - id: TC-001
    name: "Test description"
    endpoint: /api/v1/resource
    method: GET
    headers:
      Authorization: "Bearer ${API_TOKEN}"
    params:
      key: value
    body: {} # For POST/PUT
    expected:
      status: 200
      body:
        field: expected_value
    priority: high
    tags: [smoke]

sqs_assertions:
  - name: "Event name"
    queue_url: "${SQS_QUEUE_URL}"
    match_mode: json_field
    expected_fields:
      event_type: "EVENT_NAME"
    timeout_seconds: 30
```

## Error Handling

| Scenario              | Action                                     |
| --------------------- | ------------------------------------------ |
| Test file not found   | Error with suggestions for file location   |
| Invalid YAML          | Show parse error and line number           |
| Network timeout       | Mark test as ERROR with timeout message    |
| Invalid response      | Mark test as ERROR, show raw response      |
| SQS queue unavailable | Mark assertion as ERROR, continue tests    |
| Missing env variable  | Warn and use literal string                |

## Advanced Features

### Parallel Execution

For faster test runs:

```yaml
execution:
  parallel: true
  max_concurrent: 5
```

### Test Dependencies

Run tests in order with dependencies:

```yaml
test_cases:
  - id: TC-001
    name: "Create user"
    # ... creates user
    outputs:
      user_id: "$.response.id"

  - id: TC-002
    name: "Get created user"
    depends_on: TC-001
    endpoint: /api/v1/users/${outputs.user_id}
```

### Retry Logic

Retry flaky tests:

```yaml
test_cases:
  - id: TC-001
    retry:
      max_attempts: 3
      delay_ms: 1000
```