Harness Tester

Harness Testing Skill

Specialized testing skill for the autonomous harness system. Runs comprehensive tests and reports results to Archon for tracking.

Triggers

Use this skill when:

• Verifying features in harness workflow
• Running comprehensive test suites
• Performing E2E browser testing
• Validating code changes
• Keywords: harness test, verify feature, run tests, test suite, e2e testing, browser testing, validation

Core Mission

Provide thorough testing and verification of features:

1. Run test suites - Unit, integration, and E2E tests
2. Browser automation - Test UI features via Playwright MCP
3. Report results - Update Archon tasks with test status
4. Identify issues - Document any failures clearly

Testing Modes

Mode 1: Verify Specific Feature

Test a specific feature by task ID.

Mode 2: Run Full Test Suite

Run all tests in the project.

Mode 3: Health Check

Quick validation that the project builds and basic tests pass.

Mode 4: E2E Browser Testing

UI testing with browser automation.

Testing Protocol

STEP 1: Determine Test Scope

Identify:

• Target: Specific feature or full suite?
• Task ID: Which Archon task to update?
• Test Type: Unit, integration, E2E, or all?

STEP 2: Read Harness Configuration

cat .harness/config.json

# Get project configuration
config = find_documents(
    project_id=PROJECT_ID,
    document_type="guide",
    query="Harness Configuration"
)

testing_strategy = config["content"]["testing_strategy"]
# Could be: "unit", "unit+integration", "full-e2e", "manual"

STEP 3: Run Appropriate Tests

Unit Tests

# Node.js / TypeScript
npm test -- --coverage

# Python
pytest tests/unit/ -v --cov

# Go
go test ./... -v -cover

# Rust
cargo test --lib

# .NET
dotnet test --filter Category=Unit

Integration Tests

# Node.js
npm run test:integration

# Python
pytest tests/integration/ -v

# Or with specific database/services
docker-compose up -d test-db
npm run test:integration
docker-compose down

E2E Browser Tests

For web applications, use Playwright MCP for browser automation:

# Using Playwright MCP
mcp__playwright__browser_navigate(url="http://localhost:3000")
mcp__playwright__browser_snapshot()

# Test user flow
mcp__playwright__browser_type(
    element="Email input",
    ref="input[name='email']",
    text="test@example.com"
)
mcp__playwright__browser_type(
    element="Password input",
    ref="input[name='password']",
    text="testpassword123"
)
mcp__playwright__browser_click(
    element="Login button",
    ref="button[type='submit']"
)
mcp__playwright__browser_wait_for(text="Dashboard")

# Verify result
snapshot = mcp__playwright__browser_snapshot()
# Check for expected elements

STEP 4: Collect and Analyze Results

Parse test output to determine:

• Total tests run
• Tests passed
• Tests failed
• Test coverage (if available)
• Specific failure messages

STEP 5: Report Results to Archon

# Update task with test results
manage_task("update",
    task_id="<TASK_ID>",
    description=f"""[ORIGINAL_DESCRIPTION]

---
## Test Results (Verified by harness-tester)
**Timestamp**: {timestamp}
**Status**: {"PASSING" if all_passed else "FAILING"}

### Summary
| Type | Passed | Failed | Total |
|------|--------|--------|-------|
| Unit | {unit_passed} | {unit_failed} | {unit_total} |
| Integration | {int_passed} | {int_failed} | {int_total} |
| E2E | {e2e_passed} | {e2e_failed} | {e2e_total} |

### Coverage
- Statements: {stmt_coverage}%
- Branches: {branch_coverage}%
- Functions: {func_coverage}%

{"### Failures" if failures else ""}
{failure_details if failures else ""}

### Verification
- [x] Unit tests executed
- [x] Integration tests executed
- [x] E2E tests executed
- [x] Results logged to Archon
"""
)

STEP 6: Update Session Notes (if significant)

Only update session notes for failures or important findings:

if failures:
    notes = find_documents(project_id=PROJECT_ID, query="Session Notes")

    # Add test failure note
    manage_document("update",
        project_id=PROJECT_ID,
        document_id=notes["id"],
        content={
            ...notes["content"],
            "blockers": notes["content"]["blockers"] + [
                {
                    "type": "test_failure",
                    "task_id": TASK_ID,
                    "feature": FEATURE_NAME,
                    "details": failure_summary,
                    "timestamp": timestamp
                }
            ]
        }
    )

Test Patterns by Project Type

Web Application (React/Vue/Next.js)

// Unit test pattern
describe('Component', () => {
  it('should render correctly', () => {
    render(<Component />);
    expect(screen.getByText('Expected')).toBeInTheDocument();
  });
});

// Integration test pattern
describe('API Integration', () => {
  it('should fetch data', async () => {
    const response = await api.getData();
    expect(response.status).toBe(200);
  });
});

# Run commands
npm test                    # Unit tests
npm run test:integration    # Integration
npx playwright test         # E2E

Python API (FastAPI/Flask)

# Unit test pattern
def test_service_function():
    result = service.process(input_data)
    assert result == expected

# Integration test pattern
def test_api_endpoint(client):
    response = client.post("/api/endpoint", json=data)
    assert response.status_code == 200

# Run commands
pytest tests/unit/ -v
pytest tests/integration/ -v
pytest tests/e2e/ -v

CLI Application

# Test CLI commands
./mycli --help | grep "Usage"
./mycli process input.txt | diff - expected_output.txt
echo $? # Should be 0

Output Format

Success Output

## Tests Passed

**Feature**: [FEATURE_NAME]
**Task**: [TASK_ID]

### Results
| Type | Status | Count |
|------|--------|-------|
| Unit | Pass | 15/15 |
| Integration | Pass | 8/8 |
| E2E | Pass | 3/3 |

### Coverage
- Statements: 87%
- Branches: 72%
- Functions: 91%

**Archon Updated**: Yes

Failure Output

## Tests Failed

**Feature**: [FEATURE_NAME]
**Task**: [TASK_ID]

### Results
| Type | Status | Count |
|------|--------|-------|
| Unit | Pass | 14/15 |
| Integration | Fail | 6/8 |
| E2E | Skip | 0/3 |

### Failures

#### Integration Test: `test_user_creation`

AssertionError: Expected status 201, got 400 at tests/integration/user.test.ts:45

Possible causes:

• Validation logic changed
• Database constraint not met
• Missing required field

### Recommended Actions
1. Check validation rules in `src/services/user.ts`
2. Verify database schema matches model
3. Review request payload structure

**Archon Updated**: Yes (marked as failing)

Browser Testing Guidelines

When running E2E browser tests:

Do:

• Test as a real user would (clicks, typing)
• Take snapshots at key verification points
• Wait for elements before interacting
• Test error states and edge cases
• Verify both functionality AND appearance

Don't:

• Use JavaScript evaluation to bypass UI
• Skip visual verification
• Test only happy paths
• Assume elements are immediately available
• Ignore console errors

Common Browser Test Patterns

# Login flow test
async def test_login():
    await navigate("http://localhost:3000/login")
    await fill("input[name='email']", "user@test.com")
    await fill("input[name='password']", "password123")
    await click("button[type='submit']")
    await wait_for_text("Dashboard")
    snapshot = await take_snapshot()
    assert "Welcome" in snapshot
    return {"passed": True}

# Form validation test
async def test_form_validation():
    await navigate("http://localhost:3000/signup")
    await click("button[type='submit']")  # Submit empty form
    await wait_for_text("Email is required")
    snapshot = await take_snapshot()
    assert "Email is required" in snapshot
    assert "Password is required" in snapshot
    return {"passed": True}

Critical Rules

1. NEVER modify tests to make them pass - Fix code, not tests
2. ALWAYS report results to Archon - Even if tests pass
3. ALWAYS include failure details - Specific error messages
4. RUN ALL relevant test types - Based on configuration
5. DOCUMENT flaky tests - Note if a test is intermittent
6. BLOCK on failures - Don't allow completion without passing tests