Results for “benchmark-loop”

Run autonomous optimization loops that iteratively improve artifacts against evaluators using hypothesis tree refinement, without overfitting.

Clade goal-driven autonomous improvement loop (Blueprint architecture — deterministic pre/post phases + LLM supervisor/worker nodes, converges when goal met or max-iter hit). NOT the Claude Code built-in /loop (which polls a prompt on an interval like `/loop 5m /foo`) — if the user wants interval polling, route to the built-in.

Cross-model benchmark for gstack skills. (gstack)

Run a prompt or skill in this session on a recurring or variable interval (e.g. /loop 5m /foo).

Evaluate LLM outputs systematically — benchmarks, automated metrics, human preference, and regression tracking

Guides machine learning development with experiment tracking, hyperparameter optimization, model registry, and MLOps pipeline integration.

Designs and reviews feedback loops for AI agents to ensure goals are machine-decidable, loops are damped, and human judgment is preserved.

Run a Codex prompt repeatedly on a fixed interval. Use for "/loop", "run this every N minutes/hours", "poll X every 5 minutes", "repeat this Codex prompt", or any recurring interval-based Codex job. This is the interval-repeat companion to the separate `schedule` skill (cron / specific-time). For a durable, always-on alternative that survives reboots and terminal exits, prefer Codex app Automations; this local CLI is a terminal fallback that only fires while its daemon process is running.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Launches a model evaluation batch with parameter collection, pre-flight checks, execution, and post-run analysis for interactive or foreground runs.

Orchestrates bounded, governed iteration loops over existing agent commands and offices, with explicit stopping conditions, checkpoints, and honest terminal states.

Run an agent loop until an exit condition is met. Use when the user says "loop", "babysit", "keep trying until", "check every X", "watch", or wants iterative autonomous execution.

Provides guidance for automatically evolving and optimizing AI agents across any domain using LLM-driven evolution algorithms. Use when building self-improving agents, optimizing agent prompts and skills against benchmarks, or implementing automated agent evaluation loops.