How do I install the Run Evals skill?

Run npx skillmds@latest add devin-axis/run-evals in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Run Evals safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: makes network calls. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Run Evals?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Run Evals free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Run Evals?

Devin-AXIS (@devin-axis) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Run Evals by devin-axis — Agent Skill

Name: Run Evals
Rating: 4.2 (5 reviews)
Author: Devin-AXIS

Run Evals

do e2e tests, run e2e, validate feature, prove it works, PR proof, frame proof, pnpm evals. Launches iPolloWork on Daytona or local Electron and runs the coded eval flows via CDP. Launch + run mechanics; the proof loop itself is the fraimz skill.

Devin-AXIS Updated Sep 9, 2026

File contents

Skill: Run Evals

Launch a real iPolloWork app and run coded eval flows against it. This skill owns launch + run; the prove/repair/verdict loop and evidence standard live in the fraimz skill — load that too for anything that ends in a verdict.

Prerequisites

daytona CLI installed and logged in (daytona login), right org selected (daytona organization use "<org-name>")
.devcontainer/ files present in the repo
Optional provider coverage: reusable secrets volume populated once with bash .devcontainer/setup-daytona-secrets-volume.sh .newtoken (never print keys; sandboxes source every /daytona-secrets/*.env before Electron starts)

Preferred path: Daytona sandbox

daytona organization use "<org-name>"
bash .devcontainer/test-on-daytona.sh <branch-or-commit> --artifacts-volume

The helper creates a fresh VNC-capable sandbox from the ipollowork-eval-vnc snapshot, mounts the secrets + pnpm-store volumes, starts XFCE/noVNC, Vite, and Electron with Daytona-safe flags, waits for CDP, then prints the CDP and noVNC URLs. --artifacts-volume mounts /daytona-artifacts served on port 8090 for published frame proof. Refresh the snapshot when dependencies change: bash .devcontainer/create-daytona-ipollowork-snapshot.sh.

Verify the endpoint before running flows:

curl -fsS "<CDP_URL>/json/list"   # must include an iPolloWork page target

If it fails, inspect /tmp/electron.log — the real success marker is Chromium's DevTools listening on ws://127.0.0.1:9825/....

If the app shows the Welcome page, use a coded onboarding flow or create /workspace/hello through the visible UI before running a workspace-dependent flow.

Run the flows

pnpm evals --list
pnpm evals --flow <flow-id> --cdp-url <printed-electron-cdp-url>
pnpm evals --all --stack den     # brings up MySQL + den-api + seed for cloud flows

The runner produces machine-checkable assertions, validated screenshots, and writes fraimz.html + report.md / report.json under evals/results/<run-id>/. If no coded flow exists for the behavior, add one in evals/flows/<id>.flow.mjs (see the fraimz skill and evals/README.md for the ctx.* API); use manual browser tools only to debug or prototype — a coded flow is the PR evidence.

Recording (motion only)

Frame proof is the default deliverable; record video only when motion matters (streaming, animations). Start with bash .devcontainer/test-on-daytona.sh <branch> --record-video --recording-name <name>, stop with daytona exec "$SANDBOX" -- 'bash .devcontainer/stop-daytona-recording.sh', download via the port-8090 artifacts URL. Details: daytona-recording-artifacts.

Local fallback

When Daytona is down or quota-limited:

pnpm install
pnpm --filter @ipollowork/app typecheck
IPOLLOWORK_ELECTRON_REMOTE_DEBUG_PORT=9826 pnpm dev   # then:
pnpm evals --flow <flow-id> --cdp-url http://127.0.0.1:9826

Report clearly whether the result came from Daytona or the local fallback — a local run is not a Daytona validation.

Teardown

daytona delete "$SANDBOX"

---name: run-evalsdescription: do e2e tests, run e2e, validate feature, prove it works, PR proof, frame proof, pnpm evals. Launches iPolloWork on Daytona or local Electron and runs the coded eval flows via CDP. Launch + run mechanics; the proof loop itself is the fraimz skill.--- # Skill: Run Evals Launch a real iPolloWork app and run coded eval flows against it. This skill owns **launch + run**; the prove/repair/verdict loop and evidence standard live in the **`fraimz` skill** — load that too for anything that ends in a verdict. ## Prerequisites - `daytona` CLI installed and logged in (`daytona login`), right org selected (`daytona organization use "<org-name>"`) - `.devcontainer/` files present in the repo - Optional provider coverage: reusable secrets volume populated once with `bash .devcontainer/setup-daytona-secrets-volume.sh .newtoken` (never print keys; sandboxes source every `/daytona-secrets/*.env` before Electron starts) ## Preferred path: Daytona sandbox ```bash daytona organization use "<org-name>" bash .devcontainer/test-on-daytona.sh <branch-or-commit> --artifacts-volume ``` The helper creates a fresh VNC-capable sandbox from the `ipollowork-eval-vnc` snapshot, mounts the secrets + pnpm-store volumes, starts XFCE/noVNC, Vite, and Electron with Daytona-safe flags, waits for CDP, then prints the CDP and noVNC URLs. `--artifacts-volume` mounts `/daytona-artifacts` served on port 8090 for published frame proof. Refresh the snapshot when dependencies change: `bash .devcontainer/create-daytona-ipollowork-snapshot.sh`. Verify the endpoint before running flows: ```bash curl -fsS "<CDP_URL>/json/list" # must include an iPolloWork page target ``` If it fails, inspect `/tmp/electron.log` — the real success marker is Chromium's `DevTools listening on ws://127.0.0.1:9825/...`. If the app shows the Welcome page, use a coded onboarding flow or create `/workspace/hello` through the visible UI before running a workspace-dependent flow. ## Run the flows ```bash pnpm evals --list pnpm evals --flow <flow-id> --cdp-url <printed-electron-cdp-url> pnpm evals --all --stack den # brings up MySQL + den-api + seed for cloud flows ``` The runner produces machine-checkable assertions, validated screenshots, and writes `fraimz.html` + `report.md` / `report.json` under `evals/results/<run-id>/`. If no coded flow exists for the behavior, add one in `evals/flows/<id>.flow.mjs` (see the `fraimz` skill and `evals/README.md` for the `ctx.*` API); use manual browser tools only to debug or prototype — a coded flow is the PR evidence. ## Recording (motion only) Frame proof is the default deliverable; record video only when motion matters (streaming, animations). Start with `bash .devcontainer/test-on-daytona.sh <branch> --record-video --recording-name <name>`, stop with `daytona exec "$SANDBOX" -- 'bash .devcontainer/stop-daytona-recording.sh'`, download via the port-8090 artifacts URL. Details: `daytona-recording-artifacts`. ## Local fallback When Daytona is down or quota-limited: ```bash pnpm install pnpm --filter @ipollowork/app typecheck IPOLLOWORK_ELECTRON_REMOTE_DEBUG_PORT=9826 pnpm dev # then: pnpm evals --flow <flow-id> --cdp-url http://127.0.0.1:9826 ``` Report clearly whether the result came from Daytona or the local fallback — a local run is not a Daytona validation. ## Teardown ```bash daytona delete "$SANDBOX" ```

Frequently asked questions