benchflow-ai
- 34 skills
- 0 followers
- 6 repo stars
- 4 hours ago last updated
- ▌ Benchflow 2 · benchflow-ai bundleRun agent benchmarks, create tasks, analyze results, and manage agents using BenchFlow. Use when asked to benchmark an AI coding agent, run a benchmark suite, create tasks, view trajectories, or compare agent performance.
- ▌ Adaptation Parity · benchflow-ai bundleAdapt an agent to run on BenchFlow (ACP) and verify it behaves identically inside the eval harness vs standalone — closing the eval↔prod gap. Use when adding a new agent to this repo or checking an existing one for eval/prod behavior parity.
- ▌ Repo Scrub · benchflow-ai bundleScan a git repository for leaked secrets and env keys, a missing LICENSE, and unclear authorship (bot or non-employee committers), then optionally fix the safe parts. Reports secret matches with values redacted, .env files tracked or untracked, secrets found in git history, license presence, and committer domains. With --fix it untracks .env, adds it to .gitignore, writes a values-blanked .env.example, and scaffolds a LICENSE; --redact-code also replaces secrets found in tracked source. Never rewrites git history automatically — it prints the command instead. Use before publishing, open-sourcing, transferring, or sharing a repository, or when asked to find leaked keys, add a license, or check IP hygiene. Trigger phrases: 'scan for secrets', 'is there an API key in this repo', 'check before open-sourcing', 'add a license', 'clean this repo before sharing'.
- ▌ Repo Report · benchflow-ai bundleGenerate a metrics report and a shareable deliverable for a git repository: lines of code, language mix, merged PRs and their complexity tiers, test-to-code ratio, untested files, functions/classes, commits, contributors, issue references, CI and reproducibility signals, and license class — plus charts over time (commits, merged PRs and active contributors per month), representative code excerpts, up to 3 sample PRs with diffs, and flags for secrets or PII. Runs offline on a local checkout and packages a PDF, an Excel workbook, JSON, CSV and SVG charts into a single zip. Use when asked to profile, summarise, or document a codebase, produce repo statistics, assess test coverage signals, or pull representative code and PR samples. Trigger phrases: 'generate a repo report', 'analyse this codebase', 'repo stats', 'summarise this repository', 'what does this codebase look like', 'pull sample code and PRs'.
- ▌ Repo Verify · benchflow-ai bundleExecute a base→gold fail→pass check on a fix pull request to confirm it contains a genuine regression test. Checks out the pre-PR commit in a throwaway git worktree, installs dependencies, applies only the PR's test changes (expects FAIL), then applies the source fix (expects PASS). A real FAIL→PASS transition returns verified:true; a build failure, an already-passing test, or a suite needing a live server/database returns verified:false with the reason. Use to confirm a PR's tests actually reproduce the bug it fixes, check whether a fix is covered by a regression test, or validate that a repository's fixes are reproducible. Trigger phrases: 'does this PR have a real regression test', 'verify this fix', 'run the fail to pass', 'check if this PR's tests reproduce the bug'.
- ▌ Benchflow · benchflow-ai bundleRun agent benchmarks, create tasks, analyze results, and manage agents using BenchFlow. Use when asked to benchmark an AI coding agent, run a benchmark suite, create tasks, view trajectories, or compare agent performance.
- ▌ Independent Review · benchflow-ai bundleIncorporate feedback from an independent code reviewer to improve your solution. The reviewer is a different agent that analyzed your work.
- ▌ Task Review · benchflow-ai bundleSkillsBench task PR review — classifies the task track (standard / research / multimodal), runs static policy checks against the track-specific rubric, benchmarks the task across oracle plus Claude and Codex (with and without skills), audits trajectories for cheating and skill invocation, and produces a `pr-N-task-timestamp-run.txt` review report alongside a `prN.zip` bundle of trajectories. Use when reviewing a SkillsBench task PR (by number, branch, or local task path), when the user asks to review a task, run benchmarks on a PR, audit a submission, classify a task as research or multimodal track, or prepare a comment to post on a SkillsBench PR.
- ▌ Task Creator · benchflow-ai bundleSkillsBench task authoring — walk a contributor from idea to submission-ready task following CONTRIBUTING.md and the task-implementation rubric. Use when the user wants to create a new SkillsBench task, scaffold a task from an existing workflow (notebook, Excel workbook, document, dataset), convert a prompt or a benchmark item into a SkillsBench task, write skills for a task, or prepare a SkillsBench PR. Pairs with `task-review` (run that as a self-check before submitting).
- ▌ Code Specialist · benchflow-ai bundleDelegate complex coding tasks to a specialist model. Use when facing algorithmic challenges, performance optimization, or tricky debugging that benefits from focused code expertise.
- ▌ Benchflow Traj Upload · benchflow-ai bundleFind a local Claude Code or Codex session, open the BenchFlow trajectory viewer, and submit it after the user reviews it. Use this skill whenever someone pastes a BenchFlow eval prize line, wants to submit / share / contribute / upload a trajectory, set up traj upload, view a session, or pick a session to send. Also use it when they mention the eval prize, benchflow-traj-upload, or "copy this to your agent".
- ▌ Benchflow Traj Upload Ops · benchflow-ai bundleOperate, test, troubleshoot, and explain `bench traj upload` for public or trusted-direct trajectory contributions, including interactive and fully specified commands, dry runs, input validation, local secret masking, trajectory reports and previews, manifest metadata, upload progress, idempotency, and production promotion checks. Use this skill when a maintainer or operator wants to test, inspect, or debug a trajectory upload; validate a trajectory, report, or manifest; or verify the public upload path end to end. For helping a contributor submit their own session, use `benchflow-traj-upload` instead.
- ▌ Benchflow Experiment Review · benchflow-ai bundleReview Benchflow or SkillsBench task-run trajectories and integration-test Benchflow code changes. Use this skill whenever the user asks to audit traj health, failed or timed-out runs, healthy pass/fail/timeout status, no-skill leakage, skill loading, reward hacking, verifier isolation, metadata completeness, token usage, timing, Daytona-vs-Docker parity, path/root handling, coverage gaps, Docker/Daytona failures, or release-readiness of benchmark data.
- ▌ Skill Creator · benchflow-aiCreate a tiny task-specific skill pack for BenchFlow self-gen smoke tests.
- ▌ Conformance Writer · benchflow-aiWrite the conformance smoke output file when the task asks for conformance.txt.
- ▌ Mesh Analysis · benchflow-ai bundleAnalyzes 3D mesh files (STL) to calculate geometric properties (volume, components) and extract attribute data. Use this skill to process noisy 3D scan data and filter debris.
- ▌
- ▌ Mpc Horizon Tuning · benchflow-aiSelecting MPC prediction horizon and cost matrices for web handling.
- ▌ Flood Detection · benchflow-aiDetect flood events by comparing water levels to thresholds. Use when determining if flooding occurred, counting flood days, aggregating instantaneous data to daily values, or classifying flood severity.
- ▌ Usgs Data Download · benchflow-aiDownload water level data from USGS using the dataretrieval package. Use when accessing real-time or historical streamflow data, downloading gage height or discharge measurements, or working with USGS station IDs.
- ▌ Integral Action Design · benchflow-aiAdding integral action to MPC for offset-free tension tracking.
- ▌ Trend Analysis · benchflow-aiDetect long-term trends in time series data using parametric and non-parametric methods. Use when determining if a variable shows statistically significant increase or decrease over time.
- ▌ Citation Management · benchflow-ai bundleComprehensive citation management for academic research. Search Google Scholar and PubMed for papers, extract accurate metadata, validate citations, and generate properly formatted BibTeX entries. This skill should be used when you need to find papers, verify citation information, convert DOIs to BibTeX, or ensure reference accuracy in scientific writing.
- ▌ Nws Flood Thresholds · benchflow-aiDownload flood stage thresholds from NWS (National Weather Service). Use when determining flood levels for USGS stations, accessing action/minor/moderate/major flood stages, or matching stations to their flood thresholds.
- ▌ Pcap Triage Tshark · benchflow-ai bundleFast workflow to inspect PCAPs and extract protocol-level details using tshark
- ▌ State Space Linearization · benchflow-aiLinearizing nonlinear dynamics around operating points for control design.
- ▌ Pca Decomposition · benchflow-aiReduce dimensionality of multivariate data using PCA with varimax rotation. Use when you have many correlated variables and need to identify underlying factors or reduce collinearity.
- ▌ Suricata Rules Basics · benchflow-aiCore building blocks of Suricata signatures and multi-condition DPI logic
- ▌ Syzkaller Build Loop · benchflow-aiFull build workflow for adding new syscall descriptions to syzkaller
- ▌
- ▌ Syz Extract Constants · benchflow-aiDefining and extracting kernel constants for syzkaller syzlang descriptions
- ▌ Contribution Analysis · benchflow-aiCalculate the relative contribution of different factors to a response variable using R² decomposition. Use when you need to quantify how much each factor explains the variance of an outcome.
- ▌ Suricata Offline Evejson · benchflow-ai bundleRunning Suricata against PCAPs offline and validating results via eve.json
- ▌ Meteorology Driver Classification · benchflow-aiClassify environmental and meteorological variables into driver categories for attribution analysis. Use when you need to group multiple variables into meaningful factor categories.