Results for “model-ranking”

Quantifies how sensitive a language model benchmark's reliability and ranking stability are to specific design choices, such as the selection of scenarios, subscenarios, examples, and few-shot prompts. Use when the user has predictions and gold and needs to compute DIoR.

Queries Hugging Face benchmark leaderboards to find the best AI models for a task, filters by device constraints, and returns a ranked comparison table with scores.

Manage machine learning models in the Agent Platform Model Registry: list, describe, upload, update, and delete models and their versions.

Fine-tune open models or Gemini models using Agent Platform infrastructure, from environment setup through data preparation, job configuration, monitoring, and deployment.

Turn model work into a production ML system with data contracts, repeatable training, measurable quality gates, deployable artifacts, and operational monitoring.

Selects the right Gemma model for a task, recommends deployment tooling (Gradio, Transformers.js, Vertex AI, MLX), and applies optimizations like MTP and QAT.

Turn model work into a production ML system with data contracts, reproducible training, quality gates, deployable artifacts, and monitoring.

Evaluates automated metrics and vision-language models on identifying granular errors in detailed image descriptions and ranking paired descriptions against human judgments, using macro F1, pairwise accuracy, Spearman rank ρ, and Kendall's τ.