Deeptoponet Eval

deeptoponet-eval

DeepTopoNet: A Framework for Subglacial Topography Estimation on the Greenland Ice Sheets — Bayu Adhi Tama et al. (2025) (arXiv:2505.23980, 2025)

What this evaluates

Evaluates a deep learning model's ability to reconstruct subglacial bed topography by fusing sparse radar ice thickness measurements with high-resolution surface elevation and ice dynamics data. It probes spatial interpolation accuracy, structural preservation of terrain features, and robustness in data-scarce glacial regions.

Datasets

• Upernavik Isstrøm — total ?; splits: train (-1), val (-1); repo https://github.com/bayuat/DeepTopoNet

Metrics

• MAE (primary) — range: other
  • Mean absolute error between predicted and reference bed elevation values across all grid cells. Lower is better.
• RMSE — range: other
  • Root mean square error of elevation predictions. Lower is better.
• R² — range: other
  • Coefficient of determination measuring the proportion of variance in bed elevation explained by the model. Higher is better.
• SSIM — range: [0, 1]
  • Structural Similarity Index Measure evaluating luminance, contrast, and structural correlation between predicted and reference grids. Higher is better.
• PSNR — range: other
  • Peak Signal-to-Noise Ratio in decibels, derived from the mean squared error of the elevation grids. Higher is better.
• TRI Relative Difference — range: percent
  • Relative Difference (%) = |TRI_predicted - TRI_BedMachine| / TRI_predicted × 100, where TRI is the root mean square of elevation differences between a cell and its 8 neighbors. Lower is better.

Input / output format

Input: Overlapping 16×16 grid patches containing four channels: surface elevation, x/y ice velocity components, surface thickening/thinning rate, and surface mass balance.

Output: A 2D grid map of predicted subglacial bed topography (elevation values in meters) matching the spatial resolution of the input patches.

Scoring recipe

def compute_metrics(pred, gold):
    mae = np.mean(np.abs(pred - gold))
    rmse = np.sqrt(np.mean((pred - gold)**2))
    ssim = compute_ssim(pred, gold)
    psnr = 10 * np.log10(max_val**2 / np.mean((pred - gold)**2))
    tri_pred = np.sqrt(np.mean((pred[1:]-pred[:-1])**2 + (pred[:,1:]-pred[:,:-1])**2))
    tri_gold = np.sqrt(np.mean((gold[1:]-gold[:-1])**2 + (gold[:,1:]-gold[:,:-1])**2))
    tri_rel_diff = np.abs(tri_pred - tri_gold) / tri_pred * 100
    return mae, rmse, ssim, psnr, tri_rel_diff

Common pitfalls

• The 80/20 train/val split is not explicitly stated as spatial, risking data leakage given the overlapping 16×16 patch training strategy and spatially correlated ice dynamics data.
• The TRI relative difference formula divides by TRI_predicted, which can produce unstable or misleading percentages when predicted ruggedness approaches zero.
• R² scores can be negative (as shown in Table 2 for baselines), indicating the model performs worse than a horizontal mean baseline, which is often misinterpreted as a standard bounded accuracy metric.

Evidence (verbatim from paper)

For the evaluation of full grid predictions, several standard metrics are employed to comprehensively measure the model’s accuracy and structural fidelity such as MAE, RMSE, Structural Similarity Index Measure (SSIM)(Wang et al., [2004]), and Peak Signal-to-Noise Ratio (PSNR)(Hore and Ziou, [2010]). Moreover, a domain-specific metric, the Terrain Ruggedness Index (TRI) (Reily Shawn et al., [1999]), is used as a quantitative measure to evaluate local variations in terrain elevation by capturing the differences between a cell and its neighboring elevation values.

Citation

@misc{tama2025deeptoponet,
  title={DeepTopoNet: A Framework for Subglacial Topography Estimation on the Greenland Ice Sheets},
  author={Bayu Adhi Tama et al. (2025)},
  year={2025},
  note={arXiv:2505.23980}
}

• arXiv: 2505.23980