L2rpn 2020 Eval

l2rpn-2020-eval

Action Set Based Policy Optimization for Safe Power Grid Management — Bo Zhou et al. (2021) (arXiv:2106.15200, 2021)

What this evaluates

Evaluates an agent's ability to safely manage power grid topology under unexpected line failures and fluctuating renewable energy generation. It probes robustness to sudden grid attacks and adaptability to changing energy mix proportions over a full year of seasonal scenarios.

Datasets

• Grid2Op (NeurIPS 2020 L2RPN) — total 24; splits: test (24)

Metrics

• total_reward (primary) — range: other
  • Sum of re-scaled rewards across 24 unseen monthly scenarios. Higher values indicate better grid management performance.

Input / output format

Input: Grid state vector (1266 dimensions for robustness, ~3800 for adaptability) encoding substations, power lines, generators, and loads.

Output: Probability distribution over topological actions (~66,918 for robustness, ~200,000+ for adaptability), from which K=100 action candidates are sampled and selected via search-based planning.

Scoring recipe

total_score = 0.0
for scenario in unseen_monthly_scenarios: # 24 total
    env = load_scenario(scenario)
    episode_reward = 0.0
    while not env.done:
        action = policy.select_action(env.state)
        next_state, r, done, info = env.step(action)
        episode_reward += r
    total_score += re_scale_reward(episode_reward)
return total_score

Common pitfalls

• The action space is extremely large (~66k–200k topological configurations), so standard RL without action-set constraints will fail to explore effectively.
• Rewards are re-scaled per environment before summation, meaning raw episode returns cannot be directly compared across tracks or without normalization.
• Evaluation covers 24 unseen monthly scenarios, so performance is highly sensitive to seasonal load and renewable generation patterns not seen during training.

Evidence (verbatim from paper)

Each submitted agent is tested in 24 unseen scenarios that cover every month of the year. The reward in each environment is re-scaled, and the total reward of 24 environments is used for ranking.

Citation

@misc{zhou2021actionset,
  title={Action Set Based Policy Optimization for Safe Power Grid Management},
  author={Bo Zhou et al. (2021)},
  year={2021},
  note={arXiv:2106.15200}
}

• arXiv: 2106.15200