Results for “metric”

Compute the F1Score metric using torchmetrics when predictions and ground-truth labels are available.

Computes the Receiver Operating Characteristic (ROC) metric using torchmetrics, supporting binary, multiclass, and multilabel tasks.

Use to design support incentives that improve behaviour without destroying the metric — pairing pay with guardrails, naming gaming modes, and choosing measures that survive Goodhart pressure. Trigger for "incentive plan", "agent bonus scheme", "SPIFF design", "pay for QA score", "what metric should we bonus", CSAT incentives, or reviewing whether a comp change is driving gaming.

Visualize training metrics, debug models with histograms, compare experiments, visualize model graphs, and profile performance with TensorBoard - Google's ML visualization toolkit

Visualize training metrics, debug models with histograms, compare experiments, visualize model graphs, and profile performance with TensorBoard - Google's ML visualization toolkit

Debug LangChain and LangGraph agents by fetching execution traces from LangSmith. Analyze agent behavior, investigate errors, and review tool calls and performance metrics.

`analysis-agent`/`task-agent`/`review-agent`: use when symptoms, logs, metrics, regressions, or incidents need cause analysis; skip when no diagnosis decision exists.

Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics

Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics

Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics

Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics

`analysis-agent`/`task-agent`/`review-agent`: primary-Skill-selected for logs, metrics, traces, alerts, SLI/SLO, or diagnostics; never task owner; skip without signal impact.

Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics

Run evaluation tasks against a NeMo Platform server using the Evaluator plugin CLI and Python SDK.

Debug agent execution by querying trace and metric tables, analyzing tool calls, durations, errors, and performance trends.

Fit SimBiology model parameters to data — fitproblem, population NLME, virtual patients, and NCA. Use when asked to fit, estimate, calibrate, or compute PK metrics.

Queries video analytics incidents, alerts, metrics, and sensor data from Elasticsearch via the VA-MCP server.

Compare coding agents head-to-head on reproducible tasks with pass rate, cost, time, and consistency metrics.

Measures distributional similarity between original GAN-generated images and their semantically manipulated counterparts using the Fréchet Inception Distance (FID) metric.

Computes Theil's U (uncertainty coefficient) between predictions and ground truth using the torchmetrics implementation, handling categorical data and NaN strategies.

Build financial models, backtest trading strategies, and analyze market data. Implements risk metrics, portfolio optimization, and statistical arbitrage. Use PROACTIVELY for quantitative finance, trading algorithms, or risk analysis.

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

Filters and prioritizes compound libraries in drug discovery using drug-likeness rules, structural alerts, complexity metrics, and a query language.

Scaffolds a program.md research program for autoresearch by auto-detecting codebase signals and interviewing for missing details.

Computes the SQuAD metric using torchmetrics, given predictions and ground truth. Use when evaluating question-answering outputs with exact match and F1 scores.

Evaluate model quality with task-appropriate metrics and systematic error analysis. Use when: (1) comparing models, (2) analyzing failures, (3) setting go/no-go thresholds. NOT for: production monitoring implementation.

Analyze Neuropixels extracellular recordings end-to-end with SpikeInterface, covering loading, preprocessing, drift correction, spike sorting, quality metrics, and unit curation.

Designs and implements RAG pipelines, covering document chunking, embedding strategies, hybrid search, answer synthesis with source attribution, and evaluation using RAGAS metrics.

Evaluates geometric consistency in text-to-video generation by measuring structural and motion coherence across camera trajectories, detecting deformation and occlusion artifacts in static scenes.

Evaluate and rank agent results by metric or LLM judge for an AgentHub session. Use when the user runs /hub:eval or asks to score, compare, or pick a winner among completed AgentHub agents.

Evaluate and rank agent results by metric or LLM judge for an AgentHub session. Use when the user runs /hub:eval or asks to score, compare, or pick a winner among completed AgentHub agents.

Evaluate RAG pipelines with Ragas — measure faithfulness, answer relevancy, context precision/recall, and noise sensitivity using LLM-as-judge metrics; run automated test suite generation with TestsetGenerator; integrate with LangChain, LlamaIndex, and CI pipelines.

Autonomous experiment loop that optimizes any file by a measurable metric. Inspired by Karpathy's autoresearch. The agent edits a target file, runs a fixed evaluation, keeps improvements (git commit), discards failures (git reset), and loops indefinitely. Use when: user wants to optimize code speed, reduce bundle/image size, improve test pass rate, optimize prompts, improve content quality (headlines, copy, CTR), or run any measurable improvement loop. Requires: a target file, an evaluation command that outputs a metric, and a git repo.

Autonomous experiment loop that optimizes any file by a measurable metric. Inspired by Karpathy's autoresearch. The agent edits a target file, runs a fixed evaluation, keeps improvements (git commit), discards failures (git reset), and loops indefinitely. Use when: user wants to optimize code speed, reduce bundle/image size, improve test pass rate, optimize prompts, improve content quality (headlines, copy, CTR), or run any measurable improvement loop. Requires: a target file, an evaluation command that outputs a metric, and a git repo.

Evaluates RAG pipelines using a filesystem-based benchmark with corpus/ and train.json, running evaluate_rag.py to tune retrieval and generation flags and interpret RAGAS metrics.

Automates sequencing quality control by running FastQC and MultiQC, extracting quality metrics, and generating publication-ready visualizations via a CLI or MCP stdio server.