Human Test

human_test() — Real Human Feedback for AI Products

AI agents cannot judge human perception, emotion, or usability. This skill lets you call real humans to test any product URL and get structured feedback back.

What it does

1. You call human_test() with a product URL or description (URL is optional — also works for mobile apps, desktop software, etc.)
2. AI auto-generates a structured test plan
3. Real human testers claim the task on the web platform
4. Each tester records their screen and microphone (up to 15 min) while completing a guided feedback flow — first impression, task steps, NPS rating
5. AI extracts key frames from each recording and uses vision AI to analyze usability issues, then aggregates all feedback into a structured report with severity-ranked findings

Setup

Option A: Hosted (zero setup)

Use the hosted version at https://human-test.work — no installation needed. Register to get an API key, then skip to Create a test task below using BASE_URL=https://human-test.work.

Option B: Self-hosted (auto-install)

human_test() can run locally. Before creating a task, check if the server is reachable:

curl -s BASE_URL/api/config

If the server is not running, install and start it:

npm i -g humantest-app
cd /tmp && humantest init --non-interactive && cd humantest && humantest start

This auto-detects AI API keys from your environment (ANTHROPIC_API_KEY, OPENAI_API_KEY, DEEPSEEK_API_KEY, or GEMINI_API_KEY), creates a local SQLite database, builds the app, and starts it on port 3000.

A default admin user is created automatically — no registration needed.

Set BASE_URL: Ask the user once for their preferred base URL. Default: http://localhost:3000

Quick start

Create a test task

curl -X POST BASE_URL/api/skill/human-test \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://your-product.com",
    "focus": "Test the onboarding flow",
    "maxTesters": 5,
    "creator": "agent-name"
  }'

Response:

{
  "taskId": "cm...",
  "status": "OPEN",
  "testPlan": { "steps": [...], "nps": true, "estimatedMinutes": 10 }
}

Check progress and get the report

curl BASE_URL/api/skill/status/<taskId>

Response (when completed):

{
  "taskId": "cm...",
  "status": "COMPLETED",
  "submittedCount": 5,
  "report": "## Executive Summary\n...",
  "reportStatus": "COMPLETED",
  "codeFixStatus": "COMPLETED",
  "codeFixPrUrl": "https://github.com/user/repo/pull/1"
}

Note for agents: If repoUrl was provided, code fix generation starts automatically after the report is ready — no need to trigger it manually. Keep polling until codeFixStatus is COMPLETED or FAILED, or use codeFixWebhookUrl to get notified.

Parameters

Async webhooks

There are two separate webhooks for the two stages:

Report webhook (webhookUrl)

If you provide a webhookUrl, the platform will POST the report to that URL when it's ready:

{
  "event": "report",
  "taskId": "...",
  "status": "COMPLETED",
  "title": "Test: example.com",
  "targetUrl": "https://example.com",
  "report": "## Executive Summary\n...",
  "completedAt": "2026-03-02T12:00:00Z"
}

Code fix webhook (codeFixWebhookUrl)

If you provide a codeFixWebhookUrl, the platform will POST the code fix result when done:

{
  "event": "code_fix",
  "taskId": "...",
  "status": "COMPLETED",
  "title": "Test: example.com",
  "targetUrl": "https://example.com",
  "codeFixStatus": "COMPLETED",
  "codeFixPrUrl": "https://github.com/user/repo/pull/1",
  "completedAt": "2026-03-02T12:30:00Z"
}

Report format (structured for AI agents)

The report is returned as a markdown string in the report field. It uses a consistent, machine-parseable structure designed for AI agents to read and act on directly — for example, to automatically file issues, create PRs, or prioritize a fix backlog.

Section structure

Every report contains these exact sections in order:

## Metadata
| Field | Value |
|-------|-------|
| Product | ... |
| URL | ... |
| Testers | N |
| Avg NPS | X.X/10 |

## Executive Summary
(3-5 sentences, most critical finding first)

## Issues
### [CRITICAL] Issue title
- **Evidence:** (specific testers and observations)
- **Impact:** (effect on users)
- **Recommendation:** (actionable fix)

### [MAJOR] Issue title
- **Evidence:** ...
- **Impact:** ...
- **Recommendation:** ...

### [MINOR] Issue title
...

## Positive Highlights
(What worked well)

## NPS Analysis
(Score breakdown, interpretation)

## Recommendations
- **P0** (fix immediately): ... (references issue)
- **P1** (fix this sprint): ...
- **P2** (next sprint): ...
- **P3** (backlog): ...

Parsing tips for agents

• Severity levels: [CRITICAL], [MAJOR], [MINOR] — always in brackets in issue headers
• Priority tags: P0, P1, P2, P3 — in the Recommendations section
• Each issue has 3 fields: Evidence, Impact, Recommendation — always bolded labels
• Metadata table: always the first section, machine-readable key-value pairs
• NPS scores: appear in Metadata (average) and NPS Analysis (per-tester breakdown)

Agent auto-fix workflow

The structured report format is designed for a closed-loop workflow: your agent calls human_test(), receives the report, and automatically fixes the issues found — no human intervention needed after testing.

Recommended flow

1. Call human_test() with your product URL (include webhookUrl to get notified)
2. Wait for the report (poll /api/skill/status/<taskId> or receive webhook)
3. Parse the ## Issues section — each issue has [SEVERITY], Evidence, Impact, and Recommendation
4. For [CRITICAL] and [MAJOR] issues, use the Recommendation field to generate targeted code fixes
5. Create commits or PRs for each fix
6. (Optional) Call human_test() again to verify the fixes

Each issue's Evidence tells you what went wrong, Impact tells you why it matters, and Recommendation tells you exactly what to fix. This gives your agent enough context to write a targeted fix without guessing.

Repo-aware code fix suggestions

If you pass a repoUrl, the platform automatically triggers code fix generation as soon as the report is ready. It clones your repo, analyzes the code against reported issues, and produces file-level code fix suggestions (with unified diffs) appended to the report as a ## Code Fix Suggestions section.

Two modes (auto-detected)

Mode 1 — Read-only: Grant GitHub user avivahe326 read access to your repo. After the report, the platform clones the repo, analyzes the code against reported issues, and appends code-level diffs to the report.

Mode 2 — Developer access: Grant avivahe326 write access. Same as Mode 1, plus: creates a branch human-test/fixes-<taskId>, applies the diffs, pushes, and opens a PR. The PR URL is returned in the webhook payload as codeFixPrUrl and in the status API.

Example with repoUrl

curl -X POST BASE_URL/api/skill/human-test \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://your-product.com",
    "focus": "Test the checkout flow",
    "repoUrl": "https://github.com/your-org/your-repo",
    "repoBranch": "main",
    "webhookUrl": "https://your-server.com/webhook",
    "codeFixWebhookUrl": "https://your-server.com/code-fix-webhook"
  }'

Links

• Web platform: https://human-test.work
• GitHub: https://github.com/avivahe326/humantest