Model Training & Fine-tuning Agent Skills

Computes the Receiver Operating Characteristic (ROC) metric using torchmetrics, supporting binary, multiclass, and multilabel tasks.

Computes the HARE Score, an entity- and relation-centric metric for evaluating machine-generated histopathology reports against ground truth, using GatorTronS+SapBERT embeddings and relation F1.

Evaluates computational throughput and real-time efficiency of embedded CPU and GPU platforms by measuring peak FLOPS via a matrix rotation kernel and assessing inference latency and power consumption on a robotic vision pipeline.

Computes the SQuAD metric using torchmetrics, given predictions and ground truth. Use when evaluating question-answering outputs with exact match and F1 scores.

Scores generated text for morphosyntactic well-formedness by measuring how closely it adheres to language-specific dependency rules extracted from treebanks.

Evaluates text-to-image generation models by decomposing assessment into five specialized skills (object presence, count, spatial relations, scale, and text rendering) and open-ended prompts, producing interpretable binary scores with visual and textual explanations.

Evaluates the faithfulness of abstractive summaries by verifying if factual claims (masked as cloze questions) in the reference summary can be correctly answered using only the generated summary, compared against a gold-standard answer derived from the source context.

Evaluates how well internal model representations (hidden states) predict token-level information importance in summarization tasks, using NDCG@10 and Spearman's rank correlation.

Evaluates speaker verification models by computing true positive rate at fixed false positive rate thresholds, probing embedding space separation of same-speaker versus different-speaker pairs.

Benchmarks LLM agents fine-tuned with the Agent Data Protocol across software engineering, web browsing, OS/database tool use, and reasoning tasks, reporting unit test pass rates and task success rates.

Evaluates speech language models on beyond-semantic speech attributes such as dialect comprehension, multi-turn context memory, emotion perception, age-aware response generation, and non-verbal cue handling, reporting accuracy and judge-based scores.

Provides expert guidance on PyTorch Fully Sharded Data Parallel (FSDP) training, covering parameter sharding, mixed precision, CPU offloading, and FSDP2.

Fixes common PyTorch bugs including percentile calculations, LayerNorm for Conv1d, and buffer edge cases in reinforcement learning and neural network code.

Train, evaluate, export, and run inference for pose classification models using ST-GCN on skeleton keypoint sequences.

Profile a DeepStream pipeline with Nsight Systems and derive its configs from the measurement.

Create Earth2Studio diagnostic model wrappers for single-step data transformations, including simple derived diagnostics, packaged AutoModel diagnostics, and generative or diffusion diagnostics.

Provides expert guidance for distributed training with DeepSpeed, covering ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, and sparse attention.

Provides battle-tested PyTorch training recipes for LLMs, vision, diffusion, and biomedical domains, covering training loops, optimizer selection, LR scheduling, mixed precision, and debugging.

Compress large language models by 40-60% with minimal accuracy loss using one-shot pruning techniques like Wanda and SparseGPT, enabling faster inference and deployment on constrained hardware.

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality.

Compress large language models using knowledge distillation from teacher to student models, covering temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.

Fine-tune Gemma models locally using QLoRA, Unsloth, or TRL for SFT, DPO, and reward modeling, with dataset preparation and conversion to GGUF or LiteRT-LM.

Trains language models with Direct Preference Optimization using preference pairs, covering DPOTrainer setup, dataset preparation, and beta tuning for stable preference learning without explicit reward models.

Replaces GIoU with Complete IoU (CIoU) loss in PyTorch object tracking or detection tasks, combining overlap area, center-point distance, and aspect-ratio similarity for improved bounding-box regression.

Explains machine learning model predictions using SHAP values, covering feature importance, visualizations, debugging, bias analysis, and production deployment.

High-performance toolkit for genomic interval analysis in Rust with Python bindings. Use when working with genomic regions, BED files, coverage tracks, overlap detection, tokenization for ML models, or fragment analysis in computational genomics and machine learning applications.

Simulate and analyze quantum mechanical systems, including open quantum systems, using QuTiP's solvers for master equations, Lindblad dynamics, and quantum trajectories.

Pythonic wrapper around RDKit for cheminformatics, simplifying SMILES parsing, standardization, descriptors, fingerprints, clustering, 3D conformers, and parallel processing while returning native rdkit.Chem.Mol objects.

Convert chemical structures (SMILES or RDKit molecules) into numerical representations for machine learning, covering 100+ featurizers including ECFP, MACCS, descriptors, and pretrained models like ChemBERTa, with support for QSAR modeling and virtual screening.

Connects AI assistants to the Hugging Face Hub via MCP server tools to search models, datasets, Spaces, and papers, retrieve repo details and documentation, run compute jobs, and use Gradio Spaces as AI tools.

Orchestrates AI/ML development workflows covering LLM applications, RAG systems, AI agents, ML pipelines, and observability.

Guides LLM post-training with RL using slime, a Megatron+SGLang framework for training GLM, Qwen, DeepSeek, and Llama models with GRPO, async, and multi-turn workflows.

Look up and compare LLM API pricing across Models.dev, OpenRouter, and official provider pages, including token costs and aliases.

Provides guidance on using PyTorch for deep learning, covering tensors, autograd, nn.Module, DataLoaders, and best practices.

Manages Ray clusters, jobs, Serve deployments, and distributed workloads via the Dashboard API and CLI, with discovery-first checks and safety guardrails.

Predict continuous values from temporal sequences using aeon's time series regressors, covering convolutional, deep learning, distance-based, feature-based, hybrid, interval-based, and shapelet-based approaches.