Photochem Planetary Bench Eval

photochem-planetary-bench-eval

The Open-Source Photochem Code: A General Chemical and Climate Model for Interpreting (Exo)Planet Observations — Wogan et al. (2025) (arXiv:2509.25578, 2025)

What this evaluates

Evaluates a 1D photochemical and climate model's ability to simulate atmospheric composition, photochemical networks, and radiative energy balance across diverse planetary environments. It probes whether the model can reproduce observed vertical gas profiles, cloud properties, and thermal structures without relying on unphysical surface fluxes.

Datasets

• Planetary Atmospheric Observations (Venus, Earth, Mars, Titan, Jupiter, WASP-39b) — total ?; splits: test (-1); repo https://github.com/Nicholaswogan/photochem

Metrics

• reproduce observed concentrations (primary) — range: other
  • No single scalar metric is used. Evaluation is based on visual and quantitative comparison of simulated steady-state vertical profiles, column abundances, and radiative fluxes against observational datasets. Agreement is assessed by whether the model reproduces observed concentrations at specific altitudes, matches measured solar/thermal fluxes, and aligns with reference temperature profiles.

Input / output format

Input: Initial atmospheric composition (mixing ratios/pressures), temperature profile, eddy diffusion profile, surface boundary conditions (zero-flux or specified gas fluxes/deposition velocities), radiative transfer parameters, and cloud microphysics assumptions.

Output: Steady-state vertical profiles of gas concentrations, temperature-pressure profiles, net solar and thermal radiative fluxes, and cloud properties.

Scoring recipe

simulated_profiles = run_photochem_model(input_composition, temp_profile, boundary_conditions)
observed_profiles = load_observational_data(planet)

agreement = {}
for species in key_species:
    # Compare simulated vs observed at matching altitudes
    match = compare_profiles(simulated_profiles[species], observed_profiles[species])
    agreement[species] = match

# Check energy balance
flux_match = compare_fluxes(simulated_profiles.solar_flux, observed_profiles.solar_flux)
temp_match = compare_profiles(simulated_profiles.temp, observed_profiles.temp)

# Overall evaluation is pass/fail based on reproducing observed values within expected ranges
# No automated scalar score; relies on visual inspection of plots and literature comparison

Common pitfalls

• Fixing surface gas concentrations implicitly assumes unphysical compensating fluxes that mask chemical network flaws.
• Assuming a single aerosol radius instead of a realistic altitude-dependent size distribution leads to unrealistic optical properties and energy balance errors.
• Using outdated or incomplete opacity cross-sections without tuning for specific atmospheric windows can cause severe temperature profile deviations.

Evidence (verbatim from paper)

Overall, our nominal model reproduces the observed concentrations of CO, HCl, H2SO4, H2S, and SO at all altitudes, but fails to explain the measured abundances of SO2, H2O, OCS, S3 and S4.

Citation

@misc{wogan2025photochem,
  title={The Open-Source Photochem Code: A General Chemical and Climate Model for Interpreting (Exo)Planet Observations},
  author={Wogan et al. (2025)},
  year={2025},
  note={arXiv:2509.25578}
}

• arXiv: 2509.25578