Velocity Dealiasing Eval

velocity-dealiasing-eval

A Deep Learning-based Velocity Dealiasing Algorithm Derived from the WSR-88D Open Radar Product Generator — Veillette et al. (2022) (arXiv:2211.13181, 2022)

What this evaluates

Evaluates a U-Net model's ability to predict velocity fold numbers and produce dealiased radar velocity fields from folded inputs. It measures both classification accuracy for fold detection and reconstruction fidelity via velocity error metrics.

Datasets

• WSR-88D Level-II/III Radar Data — total ?; splits: train (-1), val (-1); repo https://github.com/mit-ll/unet-vda

Metrics

• velocity RMSE (primary) — range: other
  • Root Mean Square Error computed between the true velocity and the model's dealiased velocity output.
• categorical cross-entropy loss — range: [0, inf)
  • Standard categorical cross-entropy measuring classification error for fold number prediction across each image.

Input / output format

Input: 2D radar velocity images containing folded velocities, paired with corresponding Nyquist velocity thresholds.

Output: Per-pixel predicted fold number (categorical class) and the resulting dealiased velocity field.

Scoring recipe

# Velocity RMSE
rmse = np.sqrt(np.mean((true_vel - pred_dealiased_vel) ** 2))
# Categorical Cross-Entropy
loss = -np.mean(np.sum(y_true * np.log(y_pred + 1e-7)))

Common pitfalls

• Validation metrics appear better than training metrics because the validation set uses operational Nyquist velocities while the training set uses artificially lowered ones.
• Model checkpoint selection relies solely on the lowest validation RMSE, not on cross-entropy loss convergence.

Evidence (verbatim from paper)

The left panel shows categorical cross entropy loss, which describes how well the model classifies fold number across each image, and the right panel shows velocity RMSE computed between the true velocity and dealiased velocity. The training curve shows performance on the training set which includes an artificially lowered Nyquist velocity. To ensure accuracy on the target dataset, the validation curve reflects Nyquist velocities that were used operationally for the WSR-88D, which explains why the validation results are slightly better than training curves. As can be seen by the validation loss curves, the model seems to fully converge by around epoch 125. The training epoch with the lowest velocity RMSE on the validation set was selected for further testing.

Citation

@misc{veillette2022velocitydealiasing,
  title={A Deep Learning-based Velocity Dealiasing Algorithm Derived from the WSR-88D Open Radar Product Generator},
  author={Veillette et al. (2022)},
  year={2022},
  note={arXiv:2211.13181}
}

• arXiv: 2211.13181