Results for “pairwise-comparison”
19 skills
muratcankoylan Advanced Evaluation
Provides production-grade techniques for evaluating LLM outputs using LLMs as judges, covering direct scoring, pairwise comparison, bias mitigation, rubric generation, and confidence calibration.
16.9k · bundle
dvy1987 Eval Judge
Score LLM and agent outputs using LLM-as-judge techniques — direct scoring against rubrics or pairwise comparison between two outputs. Includes built-in bias mitigation for position bias, length bias, and self-enhancement bias. Load when the user asks to score an output, judge a response, evaluate against a rubric, compare two outputs, do direct scoring, run pairwise comparison, or says "rate this", "which response is better", "score this against the rubric", "judge this output", "LLM as judge this". Sub-skill of eval-output orchestrator.
3 · bundle
More results
qhjqhj00 Posh
Evaluates automated metrics and vision-language models on identifying granular errors in detailed image descriptions and ranking paired descriptions against human judgments, using macro F1, pairwise accuracy, Spearman rank ρ, and Kendall's τ.
3
qhjqhj00 Accuracy
Evaluates an AI judge system's pairwise ranking accuracy on generated commit messages against a heuristic ground truth from five automatic text metrics, using the MCMD dataset.
3
qhjqhj00 Polos
Scores generated image captions against reference captions and source images using the Polos metric, which is trained to align with human judgments and probes hallucination robustness and open-vocabulary evaluation.
3
qhjqhj00 C2c Eval
Benchmarks language model agents on the C2C multi-agent negotiation task, reporting win rate across starting positions.
3
qhjqhj00 Visor
Evaluates text-to-image models on spatial relationship accuracy using the VISOR metric, separating object detection from spatial correctness to reveal biases like object priority and merging.
3
qhjqhj00 Mdad
Quantifies the minimum accuracy gap needed between two models for a sampled micro-benchmark to reliably preserve their ranking, using the MDAD metric from Yauney et al. (2025).
3
timlai666 Pymc Bayesian Modeling
Bayesian modeling with PyMC. Build hierarchical models, MCMC (NUTS), variational inference, LOO/WAIC comparison, posterior checks, for probabilistic programming and inference.
1 · bundle
vvieira010-pixel Curriculum Crosswalk
Compares two or more band-tagged frameworks and produces a framework-neutral topic matrix showing coverage and gaps across all inputs, plus an optional reference-centric PLC crosswalk document when a reference framework is supplied.
0
nvidia Tao Train Visual Changenet
Trains, evaluates, exports, and runs inference for Visual ChangeNet models used in AOI defect detection, comparing image pairs for PASS/NO_PASS classification or change-segmentation masks.
2.2k · bundle
qhjqhj00 Menli
Evaluates the robustness and alignment with human judgment of reference-based and reference-free evaluation metrics for machine translation and summarization, particularly under adversarial conditions.
3
lingxling Matchms
Process and analyze mass spectrometry data with the Matchms Python library, including importing spectra, filtering peaks, calculating similarity scores, and building reproducible analytical workflows.
253 · bundle
jrennie99-glitch Prime Radiant
Mathematical AI interpretability with sheaf cohomology, spectral analysis, causal inference, and hallucination prevention
0
jiachen-t-wang Coyo 700m Image Text Pair Dataset Github Kakaobrain Coyo 700
COYO-700M: Image-Text Pair Dataset
6
gabrielmoreira Polars Bio
Perform fast genomic interval operations (overlap, nearest, merge, coverage, cluster, complement, subtract, count-overlaps), multi-format bioinformatics I/O, DataFusion SQL, and pileup on Polars DataFrames via the polars-bio library, serving as a scalable alternative to bioframe and bedtools.
17 · bundle
qhjqhj00 Spice
Evaluates image captions by converting them into scene graphs and computing an F-score over semantic propositions, measuring how well a generated caption captures the meaning of an image compared to human references.
3
qhjqhj00 Eas
Validates the Emotional Attitude Score (EAS) metric by measuring its consistency with human judgment on word-level sentiment polarity, using the AmbGIMT dataset and pairwise score comparisons.
3
kursku Advanced Evaluation
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
55 · bundle