yogsoth-ai Weight Perturbation
Perturb weights to test gap-ranking stability, output a stability verdict.
HARD-GATE
Pipeline
- Precheck: Verify the weight vector is normalized; verify the scoring matrix dimensions match the weight vector length
- Baseline ranking computation: Weight-sum the scoring matrix with the baseline weights to obtain the baseline ranking
- Perturbation scenario generation: Apply +20% and -20% perturbations to each dimension separately (re-normalizing afterward), generating 2×n perturbation scenarios
- Re-compute rankings: Compute a new ranking for each perturbation scenario
- Compare change magnitude: Count the number of gaps whose ranking changed in each scenario; compute Kendall τ correlation against the baseline ranking
- Stability verdict: stable (all scenarios τ ≥ 0.8) / sensitive (any scenario 0.5 ≤ τ < 0.8) / unstable (any scenario τ < 0.5)
- Output: Return a PerturbationReport object
Output Format
{
"baseline_ranking": ["gap_003", "gap_001", "gap_002"],
"perturbation_scenarios": [
{
"scenario_id": "importance_+20%",
"perturbed_weights": { "importance": 0.48, "feasibility": 0.18, "novelty": 0.17, "impact": 0.17 },
"ranking": ["gap_003", "gap_001", "gap_002"],
"kendall_tau": 1.0,
"rank_changes": 0
}
],
"min_kendall_tau": 0.87,
"stability_verdict": "stable",
"sensitive_dimensions": [],
"summary": "Stability summary (2-3 sentences)"
}