Visual Rl Rectification Eval

visual-rl-rectification-eval

Mode-Dependent Rectification for Stable PPO Training — Mohamad, Ponzio, and Descombes (2026) (arXiv:2602.05619, 2026)

What this evaluates

Evaluates the stability and generalization of PPO policies trained with mode-dependent layers (BatchNorm, dropout) across visual reinforcement learning environments. It probes whether a deterministic rectification phase prevents reward collapse and aligns training-evaluation dynamics compared to standard training modes.

Datasets

• Procgen — total 1000; splits: train (500), test (500)
• Histopathology Patch-Localization — total ?; splits: train (256), train (1024)
• Natural Image Patch-Localization — total ?; splits: train (256), train (1024)

Metrics

• normalized reward (%) (primary) — range: percent
  • Episode return divided by the optimal policy reward, multiplied by 100 to express performance as a percentage of the theoretical maximum.
• average episode return — range: other
  • Mean sum of rewards collected per episode across test environments, reported as Score.

Input / output format

Input: RGB image observations from the environment, processed through a shallow ResNet-18 backbone initialized with ImageNet weights.

Output: Action distribution sampled via the PPO policy network (discrete or continuous depending on the environment).

Scoring recipe

def compute_metrics(returns, optimal_rewards):
    normalized_reward_pct = (returns / optimal_rewards) * 100
    avg_episode_return = np.mean(returns)
    return normalized_reward_pct, avg_episode_return

Common pitfalls

• Standard BatchNorm training causes severe reward collapse on high-variability visual tasks, making direct baseline comparisons impossible without rectification.
• Evaluation must strictly separate training-mode and evaluation-mode statistics for mode-dependent layers, as their non-stationary distributions diverge during PPO optimization.
• Procgen results are averaged over 500 easy levels for training and 500 unseen levels for testing; mixing these splits invalidates generalization claims.

Evidence (verbatim from paper)

Figure 3 (top) reports performance on the patch-localization tasks, expressed as a percentage of the optimal policy reward. Figure 3 (bottom) reports performance on the Progen benchmark, measured as average episode return.

Citation

@misc{mohamad2026modedependentrectification,
  title={Mode-Dependent Rectification for Stable PPO Training},
  author={Mohamad, Ponzio, and Descombes (2026)},
  year={2026},
  note={arXiv:2602.05619}
}

• arXiv: 2602.05619