Results for “model-evaluation”

Every metric encodes an opinion about which mistake hurts.

Design, tune, and evaluate production RAG pipelines with deterministic tools for chunking, pipeline design, and retrieval evaluation.

MCP (Model Context Protocol) server build and evaluation guide, including local conventions for tool surfaces, config, and testing

Evaluate model quality with task-appropriate metrics and systematic error analysis. Use when: (1) comparing models, (2) analyzing failures, (3) setting go/no-go thresholds. NOT for: production monitoring implementation.

Entrena modelos de machine learning con Scikit-learn, LightGBM y XGBoost, desde un baseline hasta un modelo productivo con validación robusta y explicabilidad.

Turns model work into a production ML system with data contracts, reproducible training, quality gates, deployable artifacts, and monitoring.

Creates and manages AI agent teams for complex engineering tasks, with model routing, cost optimization, and performance evaluation.

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware, with backend selection between vLLM, Transformers, and accelerate.

Explain and apply core ML/AI concepts — model types, training pipelines, evaluation metrics, and neural architectures.

Machine learning in Python with scikit-learn. Use for classification, regression, clustering, model evaluation, and ML pipelines.

Turn model work into a production ML system with data contracts, reproducible training, quality gates, deployable artifacts, and monitoring.

Train, evaluate, export, quantize, and run inference for a Deformable DETR 2D object detection model using TAO, with deformable attention for efficient multi-scale feature processing.

Recommend model families and validation strategy based on data, constraints, and objective. Use when: (1) choosing algorithms, (2) balancing bias/variance, (3) planning benchmark baselines. NOT for: final legal/compliance sign-off.

通用跨模型审查：Review LLM 对任意研究制品进行独立评审，输出结构化评分、wiki 实体映射与改进建议

Models threats for a service, feature, endpoint, integration, or architecture: assets, attackers, boundaries, flows, and abuse cases.

Simulates meiotic recombination to produce offspring genomes from parent pairs, modeling Mendelian segregation, de novo mutation, sex determination, trait inference, and clinical evaluation against a disease registry.

Build evaluation frameworks for agent systems, covering rubric design, test set creation, and automated evaluation pipelines.

Evaluate LLM outputs systematically — benchmarks, automated metrics, human preference, and regression tracking

Build evaluation frameworks for agent systems with deterministic checks, regression suites, multi-dimensional rubrics, quality gates, production monitoring, and outcome measurement.

Manage machine learning models in the Agent Platform Model Registry: list, describe, upload, update, and delete models and their versions.

Provides production-grade techniques for evaluating LLM outputs using LLMs as judges, covering direct scoring, pairwise comparison, bias mitigation, rubric generation, and confidence calibration.

"Make it interpretable" is four different requests.

TRAK: Attributing Model Behavior at Scale

Cross-model benchmark for gstack skills. (gstack)

Build and improve AI or probabilistic software through evaluation-driven development. Use for LLM applications, agents, prompts, RAG, tool use, classifiers, model migrations, quality regressions, golden datasets, LLM-as-judge rubrics, benchmarks, or requests to add evals and measurable release gates. Pair with TDD for deterministic code; do not use as the primary guide for ordinary unit testing without model behavior.

Production machine-learning engineering workflow for data contracts, reproducible training, model evaluation, deployment, monitoring, and rollback. Use when building, reviewing, or hardening ML systems beyond one-off notebooks.

Guides the creation of high-quality MCP servers that let LLMs interact with external services through well-designed tools, covering planning, implementation, testing, and evaluation.

Plan which model tier handles which work BEFORE execution begins — a high-cognition model deeply understands the problem, lays the foundations, then emits a modular plan assigning each module the cheapest tier that can safely execute it, with escalation tripwires and one-way-door protection. Advisory only: it announces "next module → tier X / model Y" at each boundary and the HUMAN switches models — harnesses like Cursor cannot switch mid-run. Load when the user asks which model to use, wants a model plan, model tiers, model-tier routing, assign models to tasks or modules, says "cheap model got stuck", "which model for this task", "cost-efficient model choice", or when implementation-plan / problem-to-plan need a model: tier column. NOT dynamic-routing (plan-path selection after failure) — this skill assigns cognition tiers to work.

Large Language Model development, training, fine-tuning, and deployment best practices.

Detect poisoned training data and backdoored models across the ML pipeline using statistical analysis, activation clustering, and spectral signatures.