Jat Rl Eval

jat-rl-eval

Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent — Gallouédéc et al. (2024) (arXiv:2402.09844, 2024)

What this evaluates

Evaluates a multi-modal transformer agent's ability to perform sequential decision-making across diverse reinforcement learning domains, including Atari games, grid-world navigation, and continuous control tasks, without task-specific fine-tuning.

Datasets

• Atari 57 — total ?; splits: test (-1)
• BabyAI — total ?; splits: test (-1)
• MuJoCo — total ?; splits: test (-1)
• Meta-World — total ?; splits: test (-1)

Metrics

• expert normalized score (primary) — range: percent
  • Episode reward divided by the average expert score for that specific task, expressed as a percentage. Domain-level results are aggregated using the interquartile mean (IQM) across tasks.

Input / output format

Input: Sequential observations (e.g., pixel frames or state vectors) and implicit task context; no explicit prompt data is required during evaluation.

Output: Discrete or continuous actions per timestep, generated autoregressively by the transformer.

Scoring recipe

def compute_normalized_score(reward, expert_avg_reward):
    return (reward / expert_avg_reward) * 100.0

def compute_iqm(scores):
    sorted_scores = sorted(scores)
    n = len(sorted_scores)
    q1_idx = n // 4
    q3_idx = 3 * n // 4
    return sum(sorted_scores[q1_idx:q3_idx]) / (q3_idx - q1_idx)

# Per task
episode_rewards = [run_episode() for _ in range(100)]
task_score = compute_normalized_score(sum(episode_rewards), expert_avg_reward)

# Per domain
domain_iqm = compute_iqm([task_score for task_score in task_scores])

Common pitfalls

• Normalization uses the paper's own expert scores, which may differ from Gato's original normalization parameters, making cross-model comparisons approximate.
• BabyAI evaluation includes 39 tasks while Gato used 46; the 7 missing tasks are unspecified and likely easier, making this a harder test scenario.
• Evaluation does not use prompt data, unlike Gato, which changes the evaluation setup and may affect task identification.

Evidence (verbatim from paper)

For each task, we collect 10 evaluation episodes and normalize by the average expert score of the dataset for this task. For the final checkpoint, we use 100 evaluation episodes. We then aggregate the results by domain. The final agent achieves a mean expert normalized interquartile mean (IQM) of 63.5%, demonstrating the network’s ability to effectively mimic expert agents across a wide range of tasks.

Citation

@misc{gallouedec2024jack,
  title={Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent},
  author={Gallouédéc et al. (2024)},
  year={2024},
  note={arXiv:2402.09844}
}

• arXiv: 2402.09844