Google Research Football Eval

google-research-football-eval

Google Research Football: A Novel Reinforcement Learning Environment — Kurach et al. (2019) (arXiv:1907.11180, 2019)

What this evaluates

Evaluates reinforcement learning agents' ability to learn multi-agent coordination, strategic decision-making, and long-horizon planning in a physics-based 3D football simulation. It probes sample efficiency, reward shaping robustness, and performance under varying opponent difficulty levels.

Datasets

• Google Research Football (Football Benchmarks) — total ?; splits: easy (-1), medium (-1), hard (-1)

Metrics

• average goal difference (primary) — range: other
  • Computed as the average number of goals scored minus goals conceded per episode. Episodes are fixed at 3000 steps (300 seconds). Results are averaged over five random seeds per benchmark and reward configuration.

Input / output format

Input: Stacked Super Mini Map representation providing observations of the environment state.

Output: Discrete action selection from the environment's action space at each timestep.

Scoring recipe

def evaluate_agent(agent, benchmark_difficulty, reward_type, n_seeds=5):
    diffs = []
    for seed in range(n_seeds):
        env = FootballEnv(difficulty=benchmark_difficulty, reward=reward_type, seed=seed)
        total_diff = 0
        for step in range(3000):
            action = agent.act(env.obs)
            obs, _, done, info = env.step(action)
            total_diff += info.get('goal_diff', 0)
            if done: break
        diffs.append(total_diff)
    return sum(diffs) / len(diffs)

Common pitfalls

• RL evaluation requires averaging over multiple random seeds (paper uses 5) due to high variance in training and episode outcomes.
• Reward shaping choice (Scoring vs. Checkpoint) drastically changes learning dynamics and final performance, especially for policy gradient methods.
• Episode length is strictly fixed at 3000 steps; truncation affects credit assignment and goal difference calculation.

Evidence (verbatim from paper)

In all benchmark experiments, we use the stacked Super Mini Map representation and the same network architecture. We consider both the Scoring and Checkpoint rewards. The tuning of hyper-parameters is done using easy scenario, and we follow the same protocol for all algorithms to ensure fairness of comparison. After tuning, for each of the six considered settings (three Football Benchmarks and two reward functions), we run five random seeds and average the results. It can be seen that the environment difficulty significantly affects the training complexity and the average goal difference.

Citation

@misc{kurach2019googleresearchfootball,
  title={Google Research Football: A Novel Reinforcement Learning Environment},
  author={Kurach et al. (2019)},
  year={2019},
  note={arXiv:1907.11180}
}

• arXiv: 1907.11180