Ln 522 Manual Tester

Paths: File paths (shared/, references/, ../ln-*) are relative to skills repo root. If not found at CWD, locate this SKILL.md directory and go up one level for repo root.

Manual Tester

Manually verifies Story AC on running code and reports structured results for the quality gate.

Purpose & Scope

• Create executable test scripts in tests/manual/ folder of target project.
• Run AC-driven checks via bash/curl (API) or puppeteer (UI).
• Save scripts permanently for regression testing (not temp files).
• Document results in Linear with pass/fail per AC and script path.
• No status changes or task creation.

When to Use

• Invoked by ln-520-test-planner after ln-521-test-researcher completes
• Research comment "## Test Research:" exists on Story (from ln-521)
• All implementation tasks in Story status = Done

Test Design Principles

1. Fail-Fast - No Silent Failures

CRITICAL: Tests MUST return 1 (fail) immediately when any criterion is not met.

Never use: print_status "WARN" + return 0 for validation failures, graceful degradation without explicit flags, silent fallbacks that hide errors.

Exceptions (WARN is OK): Informational warnings that don't affect correctness, optional features (with clear justification in comments), infrastructure issues (e.g., missing Nginx in dev environment).

2. Expected-Based Testing - The Golden Standard

CRITICAL: Tests MUST compare actual results against expected reference files, not apply heuristics or algorithmic checks.

Directory structure:

tests/manual/NN-feature/
├── samples/               # Input files
├── expected/              # Expected output files (REQUIRED!)
│   └── {base_name}_{source_lang}-{target_lang}.{ext}
└── test-*.sh

Heuristics acceptable ONLY for: dynamic/non-deterministic data (timestamps, UUIDs, tokens - normalize before comparison; JSON with unordered keys - use jq --sort-keys).

3. Results Storage

Test results saved to tests/manual/results/ (persistent, in .gitignore). Named: result_{ac_name}.{ext} or response_{ac_name}.json. Inspectable after test completion for debugging.

4. Expected File Generation

To create expected files:

1. Run test with current implementation
2. Review output in results/ folder
3. If correct: copy to expected/ folder with proper naming
4. If incorrect: fix implementation first, then copy

IMPORTANT: Never blindly copy results to expected. Always validate correctness first.

Workflow

Phase 1: Setup tests/manual structure

1. Read docs/project/runbook.md — get Docker commands, API base URL, test prerequisites, environment setup
2. Check if tests/manual/ folder exists in project root
3. If missing, create structure:
   • tests/manual/config.sh — shared configuration (BASE_URL, helpers, colors)
   • tests/manual/README.md — folder documentation (see README.md template below)
   • tests/manual/test-all.sh — master script to run all test suites (see test-all.sh template below)
   • tests/manual/results/ — folder for test outputs (add to .gitignore)
4. Add tests/manual/results/ to project .gitignore if not present
5. If exists, read existing config.sh to reuse settings (BASE_URL, tokens)

Phase 2: Create Story test script

1. Fetch Story, parse AC into Given/When/Then list (3-5 expected)
   • Check for research comment (from ln-521-test-researcher) — incorporate findings into test cases
2. Detect API vs UI (API → curl, UI → puppeteer)
3. Create test folder structure:
   • tests/manual/{NN}-{story-slug}/samples/ — input files (if needed)
   • tests/manual/{NN}-{story-slug}/expected/ — expected output files (REQUIRED for deterministic tests)
4. Generate test script: tests/manual/{NN}-{story-slug}/test-{story-slug}.sh
   • Use appropriate template: TEMPLATE-api-endpoint.sh (direct calls) or TEMPLATE-document-format.sh (async jobs)
   • Header: Story ID, AC list, prerequisites
   • Test function per AC + edge/error cases
   • diff-based validation against expected files (PRIMARY)
   • Results saved to tests/manual/results/
   • Summary table with timing
5. Make script executable (chmod +x)

Phase 3: Update Documentation

1. Update tests/manual/README.md:
   • Add new test to "Available Test Suites" table
   • Include Story ID, AC covered, run command
2. Update tests/manual/test-all.sh:
   • Add call to new script in SUITES array
   • Maintain execution order (00-setup first, then numbered suites)

Phase 4: Execute and report

1. Rebuild Docker containers (no cache), ensure healthy
2. Run generated script, capture output
3. Parse results (pass/fail counts)
4. Post Linear comment with:
   • AC matrix (pass/fail per AC)
   • Script path: tests/manual/{NN}-{story-slug}/test-{story-slug}.sh
   • Rerun command: cd tests/manual && ./{NN}-{story-slug}/test-{story-slug}.sh

Critical Rules

• Scripts saved to project tests/manual/, NOT temp files.
• Rebuild Docker before testing; fail if rebuild/run unhealthy.
• Keep language of Story (EN/RU) in script comments and Linear comment.
• No fixes or status changes; only evidence and verdict.
• Script must be idempotent (can rerun anytime).

Definition of Done

• tests/manual/ structure exists (config.sh, README.md, test-all.sh, results/ created if missing).
• tests/manual/results/ added to project .gitignore.
• Test script created at tests/manual/{NN}-{story-slug}/test-{story-slug}.sh.
• expected/ folder created with at least 1 expected file per deterministic AC.
• Script uses diff-based validation against expected files (not heuristics).
• Script saves results to tests/manual/results/ for debugging.
• Script is executable and idempotent.
• README.md updated with new test suite in "Available Test Suites" table.
• test-all.sh updated with call to new script in SUITES array.
• App rebuilt and running; tests executed.
• Verdict and Linear comment posted with script path and rerun command.

Script Templates

README.md (created once per project)

# Manual Testing Scripts

> **SCOPE:** Bash scripts for manual API testing. Complements automated tests with CLI-based workflows.

## Quick Start

```bash
cd tests/manual
./00-setup/create-account.sh  # (if auth required)
./test-all.sh                 # Run ALL test suites

Prerequisites

• Docker containers running (docker compose ps)
• jq installed (apt-get install jq or brew install jq)

Folder Structure

tests/manual/
├── config.sh          # Shared configuration (BASE_URL, helpers, colors)
├── README.md          # This file
├── test-all.sh        # Run all test suites
├── 00-setup/          # Account & token setup (if auth required)
│   ├── create-account.sh
│   └── get-token.sh
└── {NN}-{topic}/      # Test suites by Story
    └── test-{slug}.sh

Available Test Suites

Adding New Tests

1. Create script in {NN}-{topic}/test-{slug}.sh
2. Update this README (Available Test Suites table)
3. Update test-all.sh (add to SUITES array)

### test-all.sh (created once per project)

```bash
#!/bin/bash
# =============================================================================
# Run all manual test suites
# =============================================================================
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/config.sh"

echo "=========================================="
echo "Running ALL Manual Test Suites"
echo "=========================================="

check_jq
check_api

# Setup (if exists)
[ -f "$SCRIPT_DIR/00-setup/create-account.sh" ] && "$SCRIPT_DIR/00-setup/create-account.sh"
[ -f "$SCRIPT_DIR/00-setup/get-token.sh" ] && "$SCRIPT_DIR/00-setup/get-token.sh"

# Test suites (add new suites here)
SUITES=(
    # "01-auth/test-auth-flow.sh"
    # "02-translation/test-translation.sh"
)

PASSED=0; FAILED=0
for suite in "${SUITES[@]}"; do
    echo ""
    echo "=========================================="
    echo "Running: $suite"
    echo "=========================================="
    if "$SCRIPT_DIR/$suite"; then
        ((++PASSED))
        print_status "PASS" "$suite"
    else
        ((++FAILED))
        print_status "FAIL" "$suite"
    fi
done

echo ""
echo "=========================================="
echo "TOTAL: $PASSED suites passed, $FAILED failed"
echo "=========================================="
[ $FAILED -eq 0 ] && exit 0 || exit 1

config.sh (created once per project)

#!/bin/bash
# Shared configuration for manual testing scripts
export BASE_URL="${BASE_URL:-http://localhost:8080}"
export RED='\033[0;31m'
export GREEN='\033[0;32m'
export YELLOW='\033[1;33m'
export NC='\033[0m'

print_status() {
    local status=$1; local message=$2
    case $status in
        "PASS") echo -e "${GREEN}[PASS]${NC} $message" ;;
        "FAIL") echo -e "${RED}[FAIL]${NC} $message" ;;
        "WARN") echo -e "${YELLOW}[WARN]${NC} $message" ;;
        "INFO") echo -e "[INFO] $message" ;;
    esac
}

check_jq() {
    command -v jq &> /dev/null || { echo "Error: jq required"; exit 1; }
}

check_api() {
    local response=$(curl -s -o /dev/null -w "%{http_code}" "$BASE_URL/health" 2>/dev/null)
    if [ "$response" != "200" ]; then
        echo "Error: API not reachable at $BASE_URL"
        exit 1
    fi
    print_status "INFO" "API reachable at $BASE_URL"
}

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
export SCRIPT_DIR

Test Script Templates

See: references/templates/

Quick start:

cp references/templates/template-api-endpoint.sh {NN}-feature/test-{feature}.sh      # Endpoint tests
cp references/templates/template-document-format.sh {NN}-feature/test-{format}.sh    # Document tests

Reference Files

• Script format reference: prompsit-api tests/manual/ (production example)
• AC format: shared/templates/test_task_template.md (or local docs/templates/ in target project)
• Risk-based context: shared/references/risk_based_testing_guide.md
• Research findings: ln-521-test-researcher creates "## Test Research" comment on Story

Version: 1.0.0 Last Updated: 2026-01-15