Cross Continual Rl Eval

cross-continual-rl-eval

CRoSS: A Continual Robotic Simulation Suite for Scalable Reinforcement Learning with High Task Diversity and Realistic Physics Simulation — Denker et al. (2026) (arXiv:2602.04868, 2026)

What this evaluates

Evaluates continual reinforcement learning capabilities in robotic simulation, specifically measuring how well agents retain performance on previously learned tasks while learning new sequential tasks. It probes catastrophic forgetting, transfer effects, and intrinsic task difficulty across line-following, object-pushing, and reaching benchmarks.

Datasets

• CRoSS — total 293; splits: MLF (150), MPO (125), HLR (10), LLR (8); repo https://github.com/anon-scientist/continual-robotic-simulation-suite

Metrics

• average cumulated score (primary) — range: other
  • The mean total reward collected over a fixed number of evaluation episodes (10 or 20) per task. For line-following, it is normalized by episode length.
• success accuracy — range: [0, 1]
  • The fraction of evaluation episodes in which the agent successfully reaches the goal within a predefined spatial or temporal tolerance.

Input / output format

Input: Simulation observations (e.g., camera images, depth maps, or robot joint/positional states depending on the benchmark setting) and current task index/context.

Output: Discrete or continuous action vector corresponding to robot control commands (e.g., wheel velocities for differential drive, joint torques/positions for arm).

Scoring recipe

def evaluate_agent(model, task, num_episodes=20):
    total_reward = 0.0
    successes = 0
    for _ in range(num_episodes):
        obs = reset_task(task)
        ep_reward = 0.0
        done = False
        while not done:
            action = model.predict(obs)
            obs, reward, done, info = step(action)
            ep_reward += reward
            if info.get('goal_reached', False):
                successes += 1
        total_reward += ep_reward
    avg_reward = total_reward / num_episodes
    accuracy = successes / num_episodes
    return avg_reward, accuracy

Common pitfalls

• Confusing episodic cumulative reward with step-wise average reward when comparing across benchmarks.
• Evaluating only on previously seen tasks (retention) rather than also measuring forward transfer to unseen future tasks.
• Assuming performance degradation is due to task difficulty rather than catastrophic forgetting, without running the independent-task baseline control.

Evidence (verbatim from paper)

As an elementary performance measure, we use the average cumulated score per (test) episode, normalized by episode length. For each task t′<t, results from 10 exploitation-only episodes are averaged after training on task t.

Citation

@misc{denker2026cross,
  title={CRoSS: A Continual Robotic Simulation Suite for Scalable Reinforcement Learning with High Task Diversity and Realistic Physics Simulation},
  author={Denker et al. (2026)},
  year={2026},
  note={arXiv:2602.04868}
}

• arXiv: 2602.04868