Criticality Detection Accuracy

criticality-detection-accuracy

Over the Edge of Chaos? Excess Complexity as a Roadblock to Artificial General Intelligence — Teo Susnjak et al. (2024) (arXiv:2407.03652, 2024)

What this evaluates

Evaluates the ability to detect the onset of systemic instability (criticality) in simulated AI systems by monitoring performance variance across multiple benchmarks. It probes whether a derivative-based threshold can reliably flag phase transitions before functional collapse.

Datasets

• Simulated Criticality Benchmarks — total ?; splits: train (-1), test (-1); repo https://github.com/teosusnjak/AGI-and-criticality

Metrics

• percentage of correct classifications (primary) — range: percent
  • Calculated as the number of simulations where criticality is detected within 10 time steps post-actual criticality, divided by the total number of simulations, expressed as a percentage.

Input / output format

Input: Time-series performance trajectories of a simulated AI system across N benchmarks (N ∈ {2, 5, 10, 20}), along with the ground-truth criticality time step.

Output: Binary classification: 'detected' if the derivative threshold is crossed within the 10-step post-criticality window, otherwise 'not detected'.

Scoring recipe

def score(predictions, gold):
    correct = 0
    for pred_time, crit_time in zip(predictions, gold):
        if crit_time <= pred_time <= crit_time + 10:
            correct += 1
    return (correct / len(predictions)) * 100

Common pitfalls

• Detection reliability heavily depends on the number of benchmarks used; fewer benchmarks yield high variance and poor detection.
• The 10-step detection window is an arbitrary experimental choice and may not generalize to real-world continuous monitoring.
• The metric relies on simulated performance data rather than live model weights or gradients, limiting direct applicability to production LLMs.

Evidence (verbatim from paper)

For the purposes of these experiments, we selected any detection that falls immediately within 10-time steps post-criticality to be a correct positive detection. Based on this criterion, Table [2] shows the percentage of correct classifications that were achieved on both the training and test datasets across all benchmark sizes.

Citation

@misc{teosusnjak2024over,
  title={Over the Edge of Chaos? Excess Complexity as a Roadblock to Artificial General Intelligence},
  author={Teo Susnjak et al. (2024)},
  year={2024},
  note={arXiv:2407.03652}
}

• arXiv: 2407.03652