Results for “gpu-evaluation”

Evaluates RAG pipelines using a filesystem-based benchmark with corpus/ and train.json, running evaluate_rag.py to tune retrieval and generation flags and interpret RAGAS metrics.

Benchmarks a text-to-text transformer across GLUE, SuperGLUE, CNN/Daily Mail, SQuAD, and WMT, reporting GLUE average, BLEU, ROUGE-2-F, and Exact Match scores.

Build, optimize, backtest, rebalance, or analyze stock portfolios using NVIDIA-accelerated Mean-CVaR optimization with cuOpt GPU solver.

Reduces peak GPU memory in Megatron Bridge training by applying expandable segments, parallelism resizing, activation recompute, and CPU offloading constraints.

Optimize MATLAB design files for GPU Coder to generate faster CUDA code. Iteratively profiles, rewrites, and benchmarks until performance targets are met or diagnostics are resolved. Use when asked to: optimize for GPU Coder, improve GPU codegen performance, profile generated GPU/CUDA code, profile GPU MEX, fix gpuPerformanceAnalyzer diagnostics, speed up GPU MEX, reduce GPU memory transfers, improve kernel parallelism, rewrite MATLAB for CUDA, or run gpuPerformanceAnalyzer.

GPU-accelerate Python code using CuPy, Numba CUDA, Warp, cuDF, cuML, cuGraph, and other RAPIDS libraries for dramatic speedups on numerical, data, ML, graph, and simulation workloads.

Quantize large language models to 4-bit with minimal accuracy loss using GPTQ, enabling deployment of 70B+ models on consumer GPUs with 4× memory reduction and 3-4× faster inference.

Benchmarks inference latency and computational runtime of transformer models and MLX operations across Apple Silicon and NVIDIA GPU backends, with configurable input lengths and batch sizes.

`task-agent`/`review-agent`: use when CPU, memory, I/O, database, network, rendering, or cost needs measured bottleneck evidence; skip without a profiling need.

Use when a bounded textual artifact (prompt, rubric, tool description, extraction instruction) keeps underperforming and success can be measured with an evaluator, dataset, or trace set. GEPA proposes evaluator-backed candidate rewrites through a normal PR/proposal adoption gate. Do not use for vague behavior changes, governance/persona/core-memory edits, fake metrics, or problems whose first honest task is defining the evaluator or collecting data.

通用跨模型审查：Review LLM 对任意研究制品进行独立评审，输出结构化评分、wiki 实体映射与改进建议

Analyzes game AI systems in a codebase, covering behavior trees, finite state machines, GOAP, utility AI, pathfinding, steering, perception, difficulty adaptation, NPC dialogue, and AI debugging tools for Unity, Unreal, and Godot projects.

Evaluates computational throughput and real-time efficiency of embedded CPU and GPU platforms by measuring peak FLOPS via a matrix rotation kernel and assessing inference latency and power consumption on a robotic vision pipeline.

Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.

Validate process decomposition and architecture design quality before execution begins. Load when the setup-evaluator agent fires (automatic for agent-chain tasks), or when user says "evaluate this setup", "check the decomposition", "validate the architecture", "is this plan sound", "review the agent design". Catches structural errors, missing knowledge, unrealistic step ordering, and topology mismatches. Does NOT modify — only evaluates.

Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.

Measures inference latency of binarized, 8-bit, and 32-bit convolutional layers on edge devices to evaluate the efficiency and speedup of the Larq Compute Engine framework compared to standard implementations.

Evaluates machine learning classifiers on their ability to distinguish signal from background in particle physics simulations, measuring how well algorithms rank signal events above background ones using the AUC metric.

GGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.

Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware, with backend selection between vLLM, Transformers, and accelerate.

Submit and manage TAO training, evaluation, and inference jobs on SLURM GPU clusters over SSH with sbatch/srun, Pyxis/Enroot containers, and Lustre-backed storage.