Results for “equal-error-rate”

Benchmarks automatic post-editing (APE) models on WMT'18 SMT, SubEdits, and MLQE-PE datasets, reporting BLEU, ChrF, and TER scores computed with SacreBLEU and TERCOM.

Compare coding agents head-to-head on reproducible tasks with pass rate, cost, time, and consistency metrics.

Runs regression evaluations comparing agent edits against human-approved golden projects, and registers new goldens after human approval.

Score LLM and agent outputs using LLM-as-judge techniques — direct scoring against rubrics or pairwise comparison between two outputs. Includes built-in bias mitigation for position bias, length bias, and self-enhancement bias. Load when the user asks to score an output, judge a response, evaluate against a rubric, compare two outputs, do direct scoring, run pairwise comparison, or says "rate this", "which response is better", "score this against the rubric", "judge this output", "LLM as judge this". Sub-skill of eval-output orchestrator.

Every metric encodes an opinion about which mistake hurts.

Prevents AI-introduced regressions with sandbox-mode API testing, automated bug-check workflows, and patterns that catch blind spots where the same model writes and reviews code.

Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics

Validates the Emotional Attitude Score (EAS) metric by measuring its consistency with human judgment on word-level sentiment polarity, using the AmbGIMT dataset and pairwise score comparisons.

Evaluate LLM outputs systematically — benchmarks, automated metrics, human preference, and regression tracking

Evaluates speaker verification models by computing true positive rate at fixed false positive rate thresholds, probing embedding space separation of same-speaker versus different-speaker pairs.

Fixes common PyTorch bugs including percentile calculations, LayerNorm for Conv1d, and buffer edge cases in reinforcement learning and neural network code.

使用seedance2.5模型生成视频,使用 Seedance 2.5 按用户原始提示词生成视频，禁止改写提示词或切换模型，并在生成前补齐时长、比例和检索所得的必要信息后向用户确认原样透传提示词、不润色视频 prompt、不要改写后生成，或显式调用本 Skill 时使用。

Compares coding agents head-to-head on reproducible tasks, measuring pass rate, cost, time, and consistency.

Master Rust 1.75+ development with async runtime (Tokio/smol).

Audit a repo (or diff) for weak error handling — swallowed exceptions, bare catches, unhandled promise rejections, missing network timeouts/retries, and errors logged but not surfaced. Pure Claude Code, no MCP, no signup.