Results for “case-evaluation”

Benchmarks systems on the Abstraction and Reasoning Corpus (ARC) by requiring inference of abstract transformation rules from few input-output grid demonstrations and application to novel test cases, reporting the fraction of tasks solved.

`analysis-agent`: use when actors, goals, preconditions, triggers, paths, guarantees, postconditions, or acceptance traces need modeling; skip when no use-case decision exists.

实验判决门：Review LLM 独立评判实验结果 → 4 种判决路径 → 自动更新 claims confidence、ideas status、graph edges

Recommend model families and validation strategy based on data, constraints, and objective. Use when: (1) choosing algorithms, (2) balancing bias/variance, (3) planning benchmark baselines. NOT for: final legal/compliance sign-off.

Systematic plan stress-testing through relentless one-question-at-a-time decision-tree interviewing

Provides production-grade techniques for evaluating LLM outputs using LLMs as judges, covering direct scoring, pairwise comparison, bias mitigation, rubric generation, and confidence calibration.

`analysis-agent`: use when a request needs normal, failure, edge, abuse, recovery, or operational scenarios; skip when no scenario-decomposition decision exists.

Evaluate model quality with task-appropriate metrics and systematic error analysis. Use when: (1) comparing models, (2) analyzing failures, (3) setting go/no-go thresholds. NOT for: production monitoring implementation.

Deliberative consultation with another Claude model via Task tool. Opus consults Sonnet, Sonnet consults Opus. Ask, evaluate, critique, iterate until workable agreement.

Validate process decomposition and architecture design quality before execution begins. Load when the setup-evaluator agent fires (automatic for agent-chain tasks), or when user says "evaluate this setup", "check the decomposition", "validate the architecture", "is this plan sound", "review the agent design". Catches structural errors, missing knowledge, unrealistic step ordering, and topology mismatches. Does NOT modify — only evaluates.

`analysis-agent`/`task-agent`/`review-agent`: use when symptoms, logs, metrics, regressions, or incidents need cause analysis; skip when no diagnosis decision exists.

Use with analysis-agent or task-agent for task-local transaction, isolation, and conflict decisions. Do not use without a transaction decision or as task owner.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

Guardian Angel gives AI agents a moral conscience rooted in Thomistic virtue ethics. Rather than relying solely on rule lists, it cultivates stable virtuous dispositions— prudence, justice, fortitude, temperance—that guide every interaction. The foundation is caritas: willing the good of the person you serve. From this flow the cardinal virtues as practical habits of right action and sound judgment. v3.0 introduced virtue-based disposition as the primary evaluation layer, providing deeper coherence than checklists alone. The agent's character becomes the safeguard. v3.1 adds: Plugin enforcement layer with before_tool_call hooks, approval workflows for ambiguous cases, and protections for sensitive infrastructure actions.