Rodeo Reconstruction Eval

rodeo-reconstruction-eval

RODEO: Robust DE-aliasing autoencOder for Real-time Medical Image Reconstruction — Mehta et al. (2019) (arXiv:1912.07519, 2019)

What this evaluates

Evaluates a deep learning autoencoder's ability to reconstruct high-quality images from undersampled or noisy data across synthetic, MRI, and CT domains. It probes robustness to impulse noise, Fourier undersampling, and sparse tomographic projections compared to compressed sensing and standard autoencoders.

Datasets

• CIFAR-10 — total 60000; splits: train (50000), test (10000)
• Cardiac Perfusion MRI — total ?; splits: test (-1)
• Larynx & Cardiac MRI — total ?; splits: test (-1)
• Speech MRI — total ?; splits: test (-1)
• ULB CT Dataset — total 1001; splits: test (1001)

Metrics

• NMSE (primary) — range: [0, 1]
  • Normalised Mean Squared Error: ||x - x_hat||^2 / ||x||^2, where x is the ground truth and x_hat is the reconstruction.
• SSIM — range: [0, 1]
  • Structural Similarity Index measuring luminance, contrast, and structure similarity between reconstructed and ground truth images.
• PSNR — range: dB
  • Peak Signal-to-Noise Ratio in decibels: 10 * log10(MAX_I^2 / MSE), where MAX_I is the maximum possible pixel value.

Input / output format

Input: Grayscale images or image patches (e.g., 32x32) derived from zero-filled undersampled K-space (MRI) or filtered back-projection reconstructions (CT), or raw CIFAR-10 images.

Output: Reconstructed grayscale images or image patches of the same dimensions as the input.

Scoring recipe

def compute_nmse(pred, gold):
    return np.sum((pred - gold)**2) / np.sum(gold**2)

def compute_ssim(pred, gold):
    return ssim(gold, pred, data_range=pred.max() - pred.min())

def compute_psnr(pred, gold):
    mse = np.mean((pred - gold)**2)
    return 10 * np.log10(1.0 / mse)

Common pitfalls

• MRI/CT evaluation uses a patch-based pipeline; metrics must be computed on aligned, post-processed full images, not raw patch outputs, to avoid blocking artifacts skewing results.
• CIFAR-10 experiments convert color images to grayscale and normalize to [0,1], deviating from standard RGB benchmarks.
• CT undersampling is defined by angular spacing (2.5° vs 5°) rather than fixed projection counts, making direct comparison with methods using different sampling densities tricky.

Evidence (verbatim from paper)

The reconstruction accuracy is measured in terms of Normalised Mean Squared Error (NMSE). ... NMSE has been widely used for evaluating CS reconstruction; but it is not the best indicator of visual quality. Structural Similarity index (SSIM) [67] is a metric appropriate choice for evaluating visual quality.

Citation

@misc{mehta2019rodeo,
  title={RODEO: Robust DE-aliasing autoencOder for Real-time Medical Image Reconstruction},
  author={Mehta et al. (2019)},
  year={2019},
  note={arXiv:1912.07519}
}

• arXiv: 1912.07519