detect-performance-discrepancy
Purpose
Detect material score discrepancies for the same method or task across sources and identify plausible condition differences.
Input contract
required: [performance_records, method_key, task_key, metric_schema]
optional: [protocol_records, condition_schema, uncertainty_estimates]
constraints: [comparisons require aligned metric direction and declared conditions]
Procedure
- Align records by method, task, metric, and observation context.
- Quantify score differences with uncertainty and identify materially different pairs.
- Compare datasets, prompts, evaluators, budgets, and protocol conditions.
- Rank plausible explanations and retain unresolved alternatives.
Output contract
produces: [discrepancy_pairs, condition_difference_map, explanation_candidates, residual_uncertainties]
delta_fields: [findings, evidence_updates, uncertainties, open_questions]
Quality gates
- Materiality uses a declared comparison basis.
- Protocol mismatch is separated from method change.
Failure and counterexamples
Do not call rounding noise a discrepancy or infer a method improvement from non-equivalent evaluation conditions.
Provenance map
resolved: discrepancy-identificationresolved: discrepancy-analysis
