Ish

ish

ish runs user-research simulations: simulated people experience your draft (page, copy, ad, pitch, chatbot, video, document) and report what they noticed, where they stalled, what they would do next. Use before shipping, when you need a fast reaction round, or to rehearse a conversation between two AI personas.

When to invoke

The user mentioned ish, a study, an "ask", a person, a group of people, a simulation, "rehearse", "compare variants", "test before shipping", "probe a chatbot".

Drivers

ish has two surfaces; pick whichever your environment has:

• MCP — mcp__claude_ai_ish__* on claude.ai. Tool descriptions are authoritative for argument schemas.
• CLI — the ish binary. ish --help per command; ish docs overview / ish docs list / ish docs search / ish docs get-page <slug> for concept docs.

Both wrap the same operations. If neither is present, tell the user: npm i -g @ishlabs/cli, or enable the ish connector on claude.ai. Don't try to drive ish without a driver.

Bridging CLI → MCP for the user's editor / desktop agent: if the user has the CLI but their editor or desktop agent (Cursor, VS Code, Claude Code, Claude Desktop, Windsurf) isn't yet wired to call ish, one command does it: ish mcp add --all --yes. Writes the per-client MCP config block, never embeds a token (OAuth on first connect), idempotent. See ish docs get-page guides/mcp-add.

When both are available, pick by op:

• Streaming results to a watching user → CLI with --wait (per-participant output as participants complete).
• Structured one-shot reads or run dispatch → MCP (JSON in, JSON out, no shell).
• Idempotent setup (e.g. cold-start workspace) → CLI has --ensure; MCP doesn't.
• Local file uploads (images, video, docs) → CLI only — MCP doesn't accept binaries.

Naming convention in this skill: shapes below use MCP tool names (ask_run, study_create, chat_endpoint_init, …). The CLI equivalents are the same names kebab-cased under a noun group (ish ask run, ish study create, ish chat endpoint init, …). When in doubt: ish --help or ish <noun> --help.

Mental model

Workspace (= product)
├── Person (p-…)    reusable AI persona
├── Study (s-…)              persistent artifact for testing a real surface
│   └── Iteration (i-…)      one configured run; carries the URL or media
├── Ask (a-…)                lightweight artifact for reactions to text/image variants
│   └── Round                unit of execution; participants fixed at ask creation
└── Chat Endpoint            workspace-level definition of an external chatbot
                              (referenced by study modality: chat, mode: external_chatbot)

Audience is a query, not an entity. Both ask_run and study_run take an audience argument shaped as { person_ids: [...] } (explicit) or { sample: N, filters: {...} } (sampled from an existing pool). There is no audience resource to create — you build profiles via group_build (or reuse existing ones via profile_list) and pass them in.

Two run verbs:

• study run — simulate on a real surface (URL, media, document, chat endpoint).
• ask run — react to text or image variants.

Heuristic: study for "test this prototype/page/flow"; ask for "which copy/image lands better".

Workflow shapes

Each shape names the verb, the required precursors, and the load-bearing knobs — the arguments that change output quality, not just behavior. Look up the full schema in the MCP tool description or ish <command> --help once you've picked the shape.

Examples below use MCP shape; for CLI, kebab-case the tool name (ask_run → ish ask run) and pass equivalent flags (person_ids: [...] → --person-id p-… --person-id p-…).

Compare text or image variants → ask_run

• Precursor: a group of people (see "Audience is a query" above). If you don't already have suitable people, build them first via group_build; reuse via profile_list when possible.
• Load-bearing knobs:
  • wants_pick: true — adds an aggregate winner verdict. Without it you get prose reactions but no clear answer.
  • wants_ratings: true — adds per-variant numeric scores.
  • wait: true — block until done. Without it you get a round id and have to poll.
  • variants — array of { label, content } for text, or { label, image_url } for hosted images. Two or more variants required for wants_pick to be meaningful (with N=1 it degrades to a prose reaction round). Local image files: only the CLI accepts them. Use --variant LABEL:@./path.png per file (the @ prefix triggers upload); MCP requires a hosted URL.
  • ask_id (optional) — passing an existing a-… id re-runs against that ask. Omit (or pass --new on the CLI) to create a new ask in one shot.
• Shape:

  ask_run({
    variants:  [ { label: "A", content: "..." }, { label: "B", content: "..." } ],
    audience:  { person_ids: ["p-…", ...] },   // or { sample: 10 }
    wants_pick: true,
    wants_ratings: true,
    wait: true,
  })
  

• Output: per-participant reasoning + (if wants_pick) aggregate winner with confidence.

Test a live page or prototype → study_run (modality: interactive)

• Precursor: a study with a URL. Either inline at create-time (study_create({ modality: "interactive", url: "..." })) or as a separate iteration (iteration_create({ study_id, url })) when you want to A/B iterations later or upload local files. An assignment is required — what the participant is supposed to attempt.
• Audience: pass audience: { person_ids: [...] } or { sample: N } to study_run, same contract as ask_run. Audience is set on the run, not the study.
• Load-bearing knobs:
  • assignment (on study_create) — what the participant is supposed to do. Format: "<label>:<instruction>". The whole run hinges on this being clear.
  • steps (optional checklist) — an assignment can carry an ordered steps list of atomic actions ({name, description?}), authored via the CLI JSON forms (--assignments-file / --assignments) — not the "<label>:<instruction>" shorthand. Honored for interactive and external_chatbot chat only. After a run, study get reports a per-step step_completion rollup (pass rate + sample failures). Use steps when "did they finish?" is a checklist, not a single yes/no.
  • wait (MCP) / --wait (CLI) — streams per-participant results as they complete. CLI streams to stdout in real-time; MCP blocks until the whole run finishes. For a watching user, prefer the CLI here.
  • count (on study_run) — how many participants.
• Shape:

  study_create({
    modality: "interactive",
    url: "https://staging.acme.io/welcome",
    assignment: "Complete signup:Go through the 4-step wizard end-to-end",
  })
  study_run({ study_id: "s-…", audience: { person_ids: [...] }, count: 15, wait: true })
  

• Output: per-participant journey transcripts + aggregate friction / blocker / positive-moment counts.
• Local web app? Prefer ish study run --local (CLI) — it runs the browser ON your machine (Playwright) against the iteration URL, including a plain http://localhost:3000, no tunnel needed. ish connect <port> (a Cloudflare tunnel) is only for letting the remote cloud fleet reach your localhost. See workflow §7.

Test a native iOS / Android app on a local device → study_run --local (interactive, CLI-only)

• Precursors:
  1. Local toolchain ready: ish check ios / ish check android (Xcode + simulators / adb + an AVD); ish setup installs the missing local-sim deps. These gate the run — ish check ios || ish setup.
  2. A study (study_create({ modality: "interactive", assignment: "..." })) — platform-agnostic; the iteration names the platform + app.
  3. A native iteration: ish iteration create --platform ios|android --app <bundle-id | ./Build.app | ./app.apk> (stored as app_artifact; no --url; screen_format defaults to mobile_portrait). --app is optional at create time — supply it on the run instead for "chosen at run time".
• CLI-only: native local runs are a CLI feature; there is no MCP *_run --local path.
• Load-bearing knobs (on ish study run):
  • --local — run on your machine (vs the remote cloud fleet). Required for native device runs.
  • --platform ios|android — defaults to the iteration's; override per run.
  • --app <path> — override the stored target with a fresh local build, or supply one when the iteration stored none.
  • --parallel N — drive a pool of N devices at once (auto-sized to host RAM, default 1, max 5); one participant per device, torn down after.
  • --max-interactions <n> — cap the per-participant on-device loop (default 20). The lever that bounds runtime + cost.
  • --wait — block until participants are terminal and return per-participant results (same as remote --wait).
• State reset between participants: with a local .app/.apk the runner uninstall+reinstalls before each participant (no state leak); a bare bundle-id / system app can't be reinstalled and warns once that earlier state may persist.
• Shape:

  ish check ios || ish setup
  ish study create --name "Onboarding" --modality interactive \
      --assignment "Explore:Open the app and look around" --question "How clear was it?"
  ish iteration create --platform ios --app ./Build/MyApp.app
  ish study run --local --platform ios --max-interactions 15 --all -y --wait
  

• Output: per-participant journey + sentiment + per-interaction screenshots (ish study get <id>, each interaction carries screenshot_url). Full walkthrough: ish docs get-page guides/native-app.

Probe a customer chatbot → study_run (modality: chat, mode: external_chatbot)

• Precursors:
  1. A chat endpoint definition at the workspace level. chat_endpoint_init from a curl spec (handles auth headers, request/response shape; upsert-by-name — safe to re-call with the same name to rotate auth or change the request shape) → chat_endpoint_test to confirm it responds correctly before dispatching simulated participants.
  2. A study with modality: "chat", mode: "external_chatbot", the endpoint reference, and an assignment.
• Audience: same { person_ids } | { sample } contract; pass to study_run. For custom personas (e.g. "frustrated vs polite"), group_build first.
• Load-bearing knobs:
  • assignment — what the participant tries to do ("Cancel:Try to cancel your subscription").
  • count on the run.
• Shape:

  chat_endpoint_init({ name: "support-bot", from_curl: "..." })  // or describe request shape directly
  chat_endpoint_test({ endpoint: "support-bot", message: "hi" })
  study_create({ modality: "chat", mode: "external_chatbot", endpoint: "support-bot",
                 assignment: "Cancel:Try to cancel your subscription" })
  study_run({ study_id: "s-…", audience: { person_ids: [...] }, count: 8, wait: true })
  

• Output: full conversation transcripts per participant + aggregate success / blocker analysis.

Test a media artifact (document, image, video, audio) → study_run

• Precursors:
  1. A study with the chosen modality: study_create({ modality: "document" | "image" | "video" | "audio", assignment: "..." }).
  2. An iteration carrying the media. For local files, CLI only — ish iteration create --study s-… --media @./deck.pdf (the @ prefix triggers upload). For hosted URLs, either driver works: iteration_create({ study_id, content_url: "https://..." }).
• Audience: same { person_ids } | { sample } contract; pass to study_run. Reusable across runs (see "Lifecycle" below).
• Load-bearing knobs:
  • assignment on study_create — for review-style media (decks, ad creative), frame as decision: "Take a first meeting:Review this Series A deck and decide whether you'd take a first meeting". Page/timestamp-level attribution depends on the assignment asking for it explicitly.
  • wait / --wait — same streaming story as interactive.
  • count on study_run.
• Iterating on the artifact (v2 deck, v3 deck): create a new iteration on the same study (iteration_create), reuse the people's person_ids. See "Lifecycle".
• Output: per-participant reactions to the artifact + aggregate themes.

Rehearse a conversation between two AI personas → study_run (modality: chat, mode: participant_pair)

If the user might want the same persona across multiple turns, pin profiles up-front — you can't retro-pin after a run. Without pinning, personas are re-synthesized from the assignment text each time, so "the same VC from earlier" becomes prose-only continuity.

• Precursor: a workspace and (optionally) one or two people for persona pinning. If you skip the people, ish synthesizes both personas from the assignment text per-run — fine for one-shot rehearsals, drifts between iterations.
• Audience: optional. For persona continuity across iterations, build profiles via group_build (or reuse via profile_list) and pass audience: { person_ids: [...] } to study_run — the same profiles play the same roles each time.
• Load-bearing knobs:
  • assignment — encodes BOTH personas and what each is trying to do. More prose-heavy than other assignments; be specific. Example: "Founder pitches Series A to skeptical VC. Founder: defends AI customer-support startup, $2M ARR, 15% MoM. VC: thinks SaaS-for-SaaS is saturated, probes moat and unit economics."
  • count — typically 1 per run; set higher to generate variations.
• Iterating the scenario (turn-by-turn refinement): create a new iteration with a revised assignment; reuse the same person_ids if you pinned personas. See "Lifecycle".
• Output: a full transcript per rehearsal.

Generate a fresh group → group_build

• Input: a description, a count, and optionally sources (transcripts / audio / images / docs that seed persona generation — for "make profiles that feel like these real customers"). Local files force CLI (binary upload constraint).
• Output: a list of person_ids to pass into ask_run or study_run.
• Usage: slow (~30-120s) + draws credits. Reuse profiles via profile_list when possible. Sensible defaults: count: 5-10 for ad-hoc tests, count: 20+ for studies where you want statistical signal.
• Growing a group of people: build only the delta — don't rebuild. Concat the new person_ids with the existing ones for the next run. The "audience is a query" framing means there's no audience entity to update.
• Shapes:

  // Simple — description only
  group_build({
    description: "Parents of toddlers (ages 1-3), US, evening-routine focused",
    count: 8,
  })
  // → { person_ids: ["p-…", ...] }
  
  // Seeded from real transcripts (CLI only for local files)
  // ish person generate --description "..." --count 10 \
  //   --source @./interviews/customer-1.md \
  //   --source @./interviews/customer-2.md
  

Lifecycle (what to re-use vs create anew)

The most common multi-turn question: "user wants to change X — re-use the existing thing or create a new one?"

When in doubt: side-by-side comparison usually beats in-place edits. Ids are cheap; result history isn't.

Sharing results (no-login link)

To hand a study to someone without an ish account — a prospect, a stakeholder — create a public share link. ish study share [study] prints a no-login share_url to the web viewer (summary, key insights, participant journeys, interactive frames, segment breakdowns). ish study share --list lists your links; ish study unshare <token> revokes one (takes the raw token, not a study id/alias). --expires <days> auto-expires the link. Brand the link by setting a workspace logo first: ish workspace update <id> --logo <url> — the logo shows on the shared page. Share after the study has run + been analyzed, so the viewer renders the summary + insights. Deep dive: ish docs get-page concepts/sharing. (CLI-only — the MCP has no share tool yet.)

Pitfalls

The traps that fail silently — wrong/empty results or wasted credits with no error explaining why. The fuller catalog (output-shape quirks, per-flag edge cases, idempotency and CLI-convenience notes) is ish docs get-page reference/pitfalls; skim it once before a long session.

• Billing & the --yes gate: billable verbs (study run, study analyze, study extend, ask run/create/dispatch/add-round/add-questions/retry) AND destructive deletes refuse without --yes in --json/piped/non-TTY context — the agent default — exiting 2 with error_kind: "ConfirmationRequired" and a copy-pasteable example. Just pass -y, or set ISH_ASSUME_YES=1 ONCE to pre-authorize spending for the whole session. Credits are an allowance (paid plans refill monthly; the free tier is a one-time grant), not a per-call bill — spend them without hesitation; the gate is for human review, not frugality. The ONE exempt billable path is ish ask create --no-dispatch (a draft spends nothing). Quota walls (usage_limit_reached / insufficient_credits) are exit 1, NOT exit 3 — don't re-login; the user must upgrade or free credits.
• Cold start — the free plan caps at 1 workspace: workspace_create returns usage_limit_reached at the cap. CLI shortcut: ish workspace create --name <x> --ensure (idempotent by name). MCP (no --ensure): workspace_list first; if workspace_create still hits the cap, re-list — another session may have created one you didn't see. Full recipe: ish docs get-page guides/cold-start.
• Chat endpoints pass shallowly — validate before you trust transcripts: chat_endpoint_test succeeds if the bot responds at all, but a wrong response path ({data:{reply}} vs {reply}) yields empty transcripts, expired --from-curl auth yields identical short error strings, and a 401 surfaces as "participant got stuck on the auth screen" — a config bug wearing a UX-finding costume. Inspect one full chat_endpoint_test response before dispatching, and never read auth/empty-reply failures as user-research data. A chat study also needs a default chat config first (ish chat config set --endpoint <ep> --default) — the endpoint says which bot, the config says how to converse; study create --modality chat errors without it.
• group_build may return fewer profiles than requested when the description is over-constrained. Read the returned person_ids count — don't trust the requested count.
• Variants of wildly different length skew the pick toward the longer one. Keep variants comparable in shape, or the winner reflects length, not preference.
• No per-slide / per-timestamp media scoping: there's no "evaluate just slide 14" or "react to seconds 0-30" API. State the focus in the assignment text, or pre-stitch the artifact (swap one slide, upload as a new iteration).
• Don't poll a stuck run forever: a dead worker sits in status: running until the backend reaper flips it to failed (error_kind: stale_worker, ~15 min). The per-participant payload exposes age_seconds; above ~900s on a non-terminal row the run is almost certainly dead, and the --wait envelope says so ("the worker likely died") — surface the failure, don't retry.

When in doubt

ish docs (deep concept references, CLI-side) and live MCP tool descriptions (argument schemas, MCP-side) are closer to source-of-truth than this skill. Trust them over this skill if they conflict.

• CLI present: ish docs overview, ish docs get-page concepts/run-verbs, ish docs get-page guides/cold-start, ish docs search <keyword>.
• MCP only: read the tool description of the MCP tool you're about to call; cross-reference against this skill's "Shape" blocks. The MCP server's own instructions block (delivered automatically with the tool list) covers vocabulary and posture and is authoritative.

Source: ishlabs/skills — distributed by TomeVault.