Results for “mle-bench”

MMBench: Is Your Multi-modal Model an All-around Player?

Production machine-learning engineering workflow for data contracts, reproducible training, model evaluation, deployment, monitoring, and rollback. Use when building, reviewing, or hardening ML systems beyond one-off notebooks.

Turn model work into a production ML system with data contracts, reproducible training, quality gates, deployable artifacts, and monitoring.

Build production ML systems with PyTorch 2.x, TensorFlow, and modern ML frameworks, including model serving, feature engineering, A/B testing, and monitoring.

Turns model work into a production ML system with data contracts, reproducible training, quality gates, deployable artifacts, and monitoring.

Production machine-learning engineering workflow for data contracts, reproducible training, model evaluation, deployment, monitoring, and rollback. Use when building, reviewing, or hardening ML systems beyond one-off notebooks.

Build production ML systems with PyTorch 2.x, TensorFlow, and modern ML frameworks. Implements model serving, feature engineering, A/B testing, and monitoring.

Provides guidance for training Mixture-of-Experts models with long context windows, covering context parallelism sizing, selective recomputation, dispatcher choices, and practical patterns from recent experiments.

Deploys and optimizes AI/ML inference workloads on GKE, using GPUs, TPUs, and model servers.

Guides machine learning development with experiment tracking, hyperparameter optimization, model registry, and MLOps pipeline integration.

Turn model work into a production ML system with data contracts, repeatable training, measurable quality gates, deployable artifacts, and operational monitoring.

Persistent memory CLI for LLM agents. Store facts, recall past knowledge, link related memories, manage lifecycle.

Machine learning development agent for end-to-end ML workflows: data preprocessing, model training, evaluation, hyperparameter tuning, and deployment

Run multi-agent research discourse session. Use when the user requests to "start a research party" or "run a journal club".

Grill me about specs for the workflows I want to build, within this workspace.

Build production ML systems with PyTorch 2.x, TensorFlow, and modern ML frameworks. Implements model serving, feature engineering, A/B testing, and monitoring. Use PROACTIVELY for ML model deployment, inference optimization, or production ML infrastructure.

Build production ML systems with PyTorch 2.x, TensorFlow, and modern ML frameworks. Implements model serving, feature engineering, A/B testing, and monitoring.

A model in production is never just weights.

Benchmark LLM performance across tasks — latency, quality, cost comparison.

Launch, monitor, stop, and debug NeMo-RL recipes on a Kubernetes cluster using the nrl-k8s CLI, supporting ephemeral and long-lived RayCluster modes.

Run Megatron-LM (MLM) and Megatron Bridge training with mock or real data, covering correlation testing, available recipes, and multi-GPU examples.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

MLOps Engineer IA — Expert en déploiement de modèles ML (A/B testing, feature stores, MLflow, Kubeflow). Industrialise le ML du notebook à la production.

Designs and implements production-grade ML pipeline infrastructure: configures experiment tracking, creates orchestration DAGs, builds feature store schemas, deploys model registries, and automates retraining and validation workflows.

Merge multiple fine-tuned models using mergekit to combine capabilities without retraining, covering SLERP, TIES-Merging, DARE, Task Arithmetic, linear merging, and production deployment strategies.

Orchestrates group discussions between installed BMAD agents, enabling natural multi-agent conversations where each agent is a real subagent with independent thinking. Use when user requests party mode, wants multiple agent perspectives, group discussion, roundtable, or multi-agent conversation about their project.

multi-llm-review

Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.

Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.

Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.