benchflow-ai
- 19 skills
- 0 followers
- 359 repo stars
- last month last updated
- ▌ Benchflow 3 · benchflow-ai bundleRun agent benchmarks, create tasks, analyze results, and manage agents using BenchFlow. Use when asked to benchmark an AI coding agent, run a benchmark suite, create tasks, view trajectories, or compare agent performance.
- ▌ Code Specialist 2 · benchflow-ai bundleDelegate complex coding tasks to a specialist model. Use when facing algorithmic challenges, performance optimization, or tricky debugging that benefits from focused code expertise.
- ▌ Benchflow 2 · benchflow-ai bundleRun agent benchmarks, create tasks, analyze results, and manage agents using BenchFlow. Use when asked to benchmark an AI coding agent, run a benchmark suite, create tasks, view trajectories, or compare agent performance.
- ▌ Adaptation Parity · benchflow-ai bundleAdapt an agent to run on BenchFlow (ACP) and verify it behaves identically inside the eval harness vs standalone — closing the eval↔prod gap. Use when adding a new agent to this repo or checking an existing one for eval/prod behavior parity.
- ▌ Repo Scrub · benchflow-ai bundleScan a git repository for leaked secrets and env keys, a missing LICENSE, and unclear authorship (bot or non-employee committers), then optionally fix the safe parts. Reports secret matches with values redacted, .env files tracked or untracked, secrets found in git history, license presence, and committer domains. With --fix it untracks .env, adds it to .gitignore, writes a values-blanked .env.example, and scaffolds a LICENSE; --redact-code also replaces secrets found in tracked source. Never rewrites git history automatically — it prints the command instead. Use before publishing, open-sourcing, transferring, or sharing a repository, or when asked to find leaked keys, add a license, or check IP hygiene. Trigger phrases: 'scan for secrets', 'is there an API key in this repo', 'check before open-sourcing', 'add a license', 'clean this repo before sharing'.
- ▌ Repo Report · benchflow-ai bundleGenerate a metrics report and a shareable deliverable for a git repository: lines of code, language mix, merged PRs and their complexity tiers, test-to-code ratio, untested files, functions/classes, commits, contributors, issue references, CI and reproducibility signals, and license class — plus charts over time (commits, merged PRs and active contributors per month), representative code excerpts, up to 3 sample PRs with diffs, and flags for secrets or PII. Runs offline on a local checkout and packages a PDF, an Excel workbook, JSON, CSV and SVG charts into a single zip. Use when asked to profile, summarise, or document a codebase, produce repo statistics, assess test coverage signals, or pull representative code and PR samples. Trigger phrases: 'generate a repo report', 'analyse this codebase', 'repo stats', 'summarise this repository', 'what does this codebase look like', 'pull sample code and PRs'.
- ▌ Repo Verify · benchflow-ai bundleExecute a base→gold fail→pass check on a fix pull request to confirm it contains a genuine regression test. Checks out the pre-PR commit in a throwaway git worktree, installs dependencies, applies only the PR's test changes (expects FAIL), then applies the source fix (expects PASS). A real FAIL→PASS transition returns verified:true; a build failure, an already-passing test, or a suite needing a live server/database returns verified:false with the reason. Use to confirm a PR's tests actually reproduce the bug it fixes, check whether a fix is covered by a regression test, or validate that a repository's fixes are reproducible. Trigger phrases: 'does this PR have a real regression test', 'verify this fix', 'run the fail to pass', 'check if this PR's tests reproduce the bug'.
- ▌ Benchflow · benchflow-ai bundleRun agent benchmarks, create tasks, analyze results, and manage agents using BenchFlow. Use when asked to benchmark an AI coding agent, run a benchmark suite, create tasks, view trajectories, or compare agent performance.
- ▌ Independent Review · benchflow-ai bundleIncorporate feedback from an independent code reviewer to improve your solution. The reviewer is a different agent that analyzed your work.
- ▌ Task Review · benchflow-ai bundleSkillsBench task PR review — classifies the task track (standard / research / multimodal), runs static policy checks against the track-specific rubric, benchmarks the task across oracle plus Claude and Codex (with and without skills), audits trajectories for cheating and skill invocation, and produces a `pr-N-task-timestamp-run.txt` review report alongside a `prN.zip` bundle of trajectories. Use when reviewing a SkillsBench task PR (by number, branch, or local task path), when the user asks to review a task, run benchmarks on a PR, audit a submission, classify a task as research or multimodal track, or prepare a comment to post on a SkillsBench PR.
- ▌ Task Creator · benchflow-ai bundleSkillsBench task authoring — walk a contributor from idea to submission-ready task following CONTRIBUTING.md and the task-implementation rubric. Use when the user wants to create a new SkillsBench task, scaffold a task from an existing workflow (notebook, Excel workbook, document, dataset), convert a prompt or a benchmark item into a SkillsBench task, write skills for a task, or prepare a SkillsBench PR. Pairs with `task-review` (run that as a self-check before submitting).
- ▌ Code Specialist · benchflow-ai bundleDelegate complex coding tasks to a specialist model. Use when facing algorithmic challenges, performance optimization, or tricky debugging that benefits from focused code expertise.
- ▌ Benchflow Traj Upload · benchflow-ai bundleFind a local Claude Code or Codex session, open the BenchFlow trajectory viewer, and submit it after the user reviews it. Use this skill whenever someone pastes a BenchFlow eval prize line, wants to submit / share / contribute / upload a trajectory, set up traj upload, view a session, or pick a session to send. Also use it when they mention the eval prize, benchflow-traj-upload, or "copy this to your agent".
- ▌ Benchflow Traj Upload Ops · benchflow-ai bundleOperate, test, troubleshoot, and explain `bench traj upload` for public or trusted-direct trajectory contributions, including interactive and fully specified commands, dry runs, input validation, local secret masking, trajectory reports and previews, manifest metadata, upload progress, idempotency, and production promotion checks. Use this skill when a maintainer or operator wants to test, inspect, or debug a trajectory upload; validate a trajectory, report, or manifest; or verify the public upload path end to end. For helping a contributor submit their own session, use `benchflow-traj-upload` instead.
- ▌ Benchflow Experiment Review · benchflow-ai bundleReview Benchflow or SkillsBench task-run trajectories and integration-test Benchflow code changes. Use this skill whenever the user asks to audit traj health, failed or timed-out runs, healthy pass/fail/timeout status, no-skill leakage, skill loading, reward hacking, verifier isolation, metadata completeness, token usage, timing, Daytona-vs-Docker parity, path/root handling, coverage gaps, Docker/Daytona failures, or release-readiness of benchmark data.
- ▌ Skill Creator · benchflow-aiCreate a tiny task-specific skill pack for BenchFlow self-gen smoke tests.
- ▌ Conformance Writer · benchflow-aiWrite the conformance smoke output file when the task asks for conformance.txt.
- ▌ Mesh Analysis · benchflow-ai bundleAnalyzes 3D mesh files (STL) to calculate geometric properties (volume, components) and extract attribute data. Use this skill to process noisy 3D scan data and filter debris.
- ▌ Citation Management · benchflow-ai bundleComprehensive citation management for academic research. Search Google Scholar and PubMed for papers, extract accurate metadata, validate citations, and generate properly formatted BibTeX entries. This skill should be used when you need to find papers, verify citation information, convert DOIs to BibTeX, or ensure reference accuracy in scientific writing.