Results for “harness”

Use when adding a new agent harness (CLI-based coding agent) adapter to adapters. Covers capability audit, adapter scaffold, session parsing, auth detection, hooks/plugins wiring, tests, and docs.

Uses a multi-agent generator-evaluator feedback loop to build high-quality applications from a single prompt, inspired by GANs and Anthropic's harness design.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag) using standardized prompts and metrics. Supports HuggingFace, vLLM, and API backends.

Formal evaluation framework for Codex sessions implementing eval-driven development (EDD) principles

Prevent repeated AI coding-agent mistakes by turning failures into durable instructions, drift checks, regression tests, failure memory, and adoption reports tailored to the target repository.

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

Designs and governs recursive self-improvement loops where an agent mines its own failures and proposes edits to its own harness, prompts, or workflow, covering acceptance gates, diversity preservation, and the optimization ladder.

Sets up a codebase for reliable agent-driven development by making it legible (structured docs, custom lints, code graph), executable (one-command dev stack, cloud sandbox), and verifiable (e2e gate, verify-before-ship loop).

Orchestrator for agent harness work — the setup that makes AI agents follow project rules and improve when they fail. FIRES PROACTIVELY when agents misbehave, repeat mistakes, ignore instructions, skip skills, or when AGENTS.md exists but docs/harness/manifest.json is missing. Also triggers on: harness engineering, agent scaffold, agent keeps failing, agent not following instructions, make agents reliable, agents going off rails, agent forgot context, improve agent setup, self-improving agents, agents keep making mistakes, why is my agent bad, agent quality, agent setup broken, agents ignore skills, same mistake again, fix agent behavior, tune agent instructions, set up agent infrastructure, after project setup agents still bad. Routes bootstrap vs evolution. Not multi-agent topology — agent-builder.

Provider-neutral processkit MCP gateway. Use when a harness should see one processkit MCP surface while processkit keeps per-skill servers canonical and available.

Designing and evaluating AI/ML systems: prompt engineering, RAG design, LLM application patterns, AI safety, evaluation frameworks, MLOps, cost optimization. Use for AI pipelines or eval harnesses.

Orchestrate parallel AI agent sessions using dmux, a tmux pane manager for agent harnesses like Claude Code and Codex.

Provides patterns for continuous autonomous agent loops with quality gates, evals, and recovery controls.

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles.

Everything Claude Code (ECC) — agent harness configuration layer for Claude Code and other AI coding IDEs. Augments agents with skills, hooks, persistent memory, model routing, and quality gates without modifying the underlying model.

Run the Ouroboros specification-first development loop: reduce ambiguity with a Socratic interview grounded in live git data (commits, churn, contributors), freeze an immutable seed/spec, render the execution plan through spec-kit (/speckit.plan → /speckit.tasks), execute against that contract through cli-anything agent-native CLI harnesses (cli-hub, --json evidence), verify before claiming success, and keep looping until completion is actually verified. Use when the user wants spec-first clarification, git-aware interviews, immutable requirements, drift-aware implementation, harness-driven execution, or a persistent completion loop that should keep going until tests / checks / acceptance criteria pass. Triggers on: ooo, ouroboros, interview, seed, run workflow, evaluate, evolve, ooo ralph, specification first, socratic interview, git-aware interview, ambiguity reduction, execution plan, cli harness execute, persistent completion.

Expose the processkit MCP tool surface through a single stdio server. Use for harnesses that eagerly start every configured MCP server, such as Codex, while keeping per-skill MCP servers available.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Extend Hermes Agent by adding new tools (sync + async patterns), authoring in-repo skills, upgrading Hermes, and understanding s6 container supervision. Class-level umbrella for Hermes development workflows.

Extend and upgrade Hermes Agent's context compression system — StagedArchiver, knowledge fingerprinting, /uncompress command, look-ahead triggers, and schema migration patterns.

Tags harness-decomposed curriculum items (KUDs, LTs, criteria) with a school's developmental band metadata while preserving source voice and labels. Supply the band schema; the skill derives mapping rules from it.

Measurement-driven model router for Claude Code. Routes Haiku/Sonnet/Opus per task class, logs every routed task with real $ numbers, and A/B tests cheaper tiers before you trust the savings.

Provides structured guidance for building production-grade AI agent systems through context engineering, covering fundamentals, architectural patterns, operational excellence, and evaluation.

Orchestrates parallel AI agent sessions using dmux, a tmux pane manager, with patterns for research, implementation, testing, and code review across multiple harnesses.

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution on local Docker, Slurm HPC, or cloud platforms.

Mines SpecStory coding histories from any agent into a persistent corpus, surfaces reproducible workflows with corroborated evidence, and interactively forges chosen ones into skills installed across agent harnesses.

Multi-agent orchestration using dmux (tmux pane manager for AI agents). Patterns for parallel agent workflows across Claude Code, Codex, OpenCode, and other harnesses. Use when running multiple agent sessions in parallel or coordinating multi-agent development workflows.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Multi-agent orchestration using dmux (tmux pane manager for AI agents). Patterns for parallel agent workflows across Claude Code, Codex, OpenCode, and other harnesses. Use when running multiple agent sessions in parallel or coordinating multi-agent development workflows.