Results for “predictions”

Computes Theil's U (uncertainty coefficient) between predictions and ground truth using the torchmetrics implementation, handling categorical data and NaN strategies.

Provides guidance on using the Aeon library for time series forecasting, covering model selection, implementation, and evaluation.

Evaluates how well internal model representations (hidden states) predict token-level information importance in summarization tasks, using NDCG@10 and Spearman's rank correlation.

Predict continuous values from temporal sequences using aeon's time series regressors, covering convolutional, deep learning, distance-based, feature-based, hybrid, interval-based, and shapelet-based approaches.

Create Earth2Studio prognostic model wrappers that time-step weather forecasts forward, with triple-inheritance classes, tests, and documentation.

Evaluates the robustness and alignment with human judgment of reference-based and reference-free evaluation metrics for machine translation and summarization, particularly under adversarial conditions.

Computes the SQuAD metric using torchmetrics, given predictions and ground truth. Use when evaluating question-answering outputs with exact match and F1 scores.

Evaluates an AI judge system's pairwise ranking accuracy on generated commit messages against a heuristic ground truth from five automatic text metrics, using the MCMD dataset.

Explains machine learning model predictions using SHAP values, covering feature importance, visualization plots, model debugging, bias analysis, and production deployment.

Extract false-positive and false-negative gaps from VLM binary-classification-question predictions by comparing model responses against ground truth, producing a structured JSONL file and summary report for downstream root-cause analysis.

Quantifies how sensitive a language model benchmark's reliability and ranking stability are to specific design choices, such as the selection of scenarios, subscenarios, examples, and few-shot prompts. Use when the user has predictions and gold and needs to compute DIoR.