Malbec Rt Intercomparison Eval

malbec-rt-intercomparison-eval

Modeling Atmospheric Lines By the Exoplanet Community (MALBEC) version 1.0: A CUISINES radiative transfer intercomparison project — Villanueva et al. (2024) (arXiv:2402.04329, 2024)

What this evaluates

Evaluates radiative transfer models' ability to simulate exoplanet transit and direct-imaging spectra under controlled atmospheric conditions. Probes how atmospheric discretization, opacity treatments, and spectroscopic databases impact spectral predictions.

Datasets

• MALBEC Test Suite — total 12; splits: test (12); repo https://github.com/projectcuisines/malbec

Metrics

• ppm (primary) — range: other
  • Absolute difference between a model's simulated transit depth (or flux) and a reference/mean spectrum, measured in parts per million (ppm) across the 0.2–20 μm wavelength range.

Input / output format

Input: Atmospheric profiles (temperature, pressure, molecular abundances), planetary/stellar radii, spectral range (0.2–20 μm), resolution (RP=200), and test-specific configurations (layer count, opacity type, linelist source).

Output: Simulated spectrum: transit depth or reflected/emitted flux as a function of wavelength, output at the specified resolution.

Scoring recipe

def compute_spectral_diff(model_spectrum, reference_spectrum, wavelengths):
    diff = np.abs(model_spectrum - reference_spectrum)
    ppm_diff = diff * 1e6
    return {
        'max_ppm': np.max(ppm_diff),
        'mean_ppm': np.mean(ppm_diff),
        'wavelengths': wavelengths
    }

Common pitfalls

• Models must use the provided MALBEC configuration files; default settings introduce uncontrolled biases.
• Sub-layering algorithms are required for ray-tracing; omitting them causes discretization errors >500 ppm.
• Spectroscopic databases must match the atmospheric regime (e.g., HITEMP for high-temp H2-rich, not HITRAN).
• The goal is not model agreement but identifying systematic discrepancies in physical parameterizations.

Evidence (verbatim from paper)

The effects of not including sub-layering in the RT models can be quite noticeable, as shown in the middle panels of Figure 1, reaching absolute errors beyond 500 ppm at 2μm on the transit radius when no sub-layering is included in the model.

Citation

@misc{villanueva2024malbec,
  title={Modeling Atmospheric Lines By the Exoplanet Community (MALBEC) version 1.0: A CUISINES radiative transfer intercomparison project},
  author={Villanueva et al. (2024)},
  year={2024},
  note={arXiv:2402.04329}
}

• arXiv: 2402.04329