Results for “channel-evaluation”

Evaluate and rank agent results by metric or LLM judge for an AgentHub session.

Fine-tune, evaluate, run inference, and export Cosmos-Embed1 video-text embedding models for tasks like text-to-video retrieval and semantic deduplication.

Evaluates RAG pipelines using a filesystem-based benchmark with corpus/ and train.json, running evaluate_rag.py to tune retrieval and generation flags and interpret RAGAS metrics.

Evaluate and rank agent results by metric or LLM judge for an AgentHub session.

Evaluate and rank agent results by metric or LLM judge for an AgentHub session. Use when the user runs /hub:eval or asks to score, compare, or pick a winner among completed AgentHub agents.

Use `analysis-agent` to map acceptance to validation, `task-agent` to add or run bounded tests, and `review-agent` to assess proof coverage. Skip work with no material change or already-fresh complete validation.

Benchmarks language model agents on the C2C multi-agent negotiation task, reporting win rate across starting positions.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

Provides production-grade techniques for evaluating LLM outputs using LLMs as judges, covering direct scoring, pairwise comparison, bias mitigation, rubric generation, and confidence calibration.

Benchmarks LLM bias by scoring responses to automatically generated open-ended questions across sensitive attributes, producing a composite fitness score from 0 to 5.

Evaluate LLM outputs systematically — benchmarks, automated metrics, human preference, and regression tracking

Two-stage review engine with four-layer quality model for the WinDAGs meta-DAG. Receives completed node outputs and produces ReviewResult containing QualityVector. Stage 1 (Haiku) checks Floor + Wall on every node. Stage 2 (Sonnet) runs Ceiling evaluation conditionally using economic escalation formula. Enforces BC-EVAL-001 through BC-EVAL-006. Activate when operating as the Evaluator role in the meta-DAG, when reviewing node outputs, when computing quality vectors, or when deciding Stage 2 escalation.

Evaluate and rank agent results by metric or LLM judge for an AgentHub session. Use when the user runs /hub:eval or asks to score, compare, or pick a winner among completed AgentHub agents.

Build evaluation frameworks for agent systems with deterministic checks, regression suites, multi-dimensional rubrics, quality gates, production monitoring, and outcome measurement.

Evaluate and rank agent results by metric or LLM judge for an AgentHub session.

Benchmarks long-context language models across 20 sub-tasks spanning 3k–200k tokens, covering retrieval, reasoning, summarization, and instruction understanding, with exact-match accuracy as the primary metric.

Evaluates spatial realism and temporal flow consistency of AI-generated videos using embedding spaces and Fourier transforms, producing bounded STREAM-S and STREAM-T scores.

`analysis-agent`/`task-agent`/`review-agent`: use for input authority, parsing, canonicalization, bounds, state checks, writable fields, rejection, or external-response changes.