Gwrfrf Spatial Spectrum Eval

gwrfrf-spatial-spectrum-eval

Generalizable Radio-Frequency Radiance Fields for Spatial Spectrum Synthesis — Kang Yang, Yuning Chen, Wan Du (2025) (arXiv:2502.05708, 2025)

What this evaluates

This benchmark evaluates a model's ability to synthesize accurate spatial radio-frequency spectra at target transmitter locations using neighboring spectra and scene geometry. It probes both single-scene prediction accuracy and cross-scene generalization capabilities in wireless propagation environments.

Datasets

• RFID Dataset — total 6123; splits: train (-1), test (-1)
• MATLAB Dataset — total 32625; splits: train (-1), test (-1)

Metrics

• MSE (primary) — range: other
  • Mean Squared Error: average of the squared differences between predicted and ground truth spectrum values. Standard regression loss.
• LPIPS — range: [0, 1]
  • Learned Perceptual Image Patch Similarity: measures perceptual difference using features from a pre-trained network. Lower is better.
• PSNR — range: other
  • Peak Signal-to-Noise Ratio: 10 * log10(max_val^2 / MSE). Higher is better, typically measured in dB.
• SSIM — range: other
  • Structural Similarity Index Measure: evaluates luminance, contrast, and structure similarity between two images. Higher is better.

Input / output format

Input: 2D spatial spectrum images (360×90) from neighboring transmitter locations, along with scene geometry/CAD models.

Output: Predicted 2D spatial spectrum image (360×90) at the target transmitter location.

Scoring recipe

import numpy as np
from skimage.metrics import peak_signal_noise_ratio, structural_similarity

def compute_metrics(y_true, y_pred):
    mse = np.mean((y_true - y_pred) ** 2)
    psnr = peak_signal_noise_ratio(y_true, y_pred)
    ssim = structural_similarity(y_true, y_pred)
    lpips = compute_lpips(y_true, y_pred)  # Requires LPIPS library
    return {'MSE': mse, 'PSNR': psnr, 'SSIM': ssim, 'LPIPS': lpips}

Common pitfalls

• The dataset is synthetically generated via MATLAB ray tracing and not publicly hosted; users must replicate the simulation setup or use the provided NeRF² dataset.
• Evaluation settings differ significantly between single-scene (train/test on same layout) and cross-scene/generalization (train on V1, test on V2/V3 or different layouts); results are not directly comparable across these settings.
• Spectra are treated as 2D images (360×90) for metric computation, not raw 1D signal vectors.

Evidence (verbatim from paper)

Signal power prediction is evaluated as a regression task with Mean Squared Error (MSE). Since the spatial spectrum, visualized in Figure 2(b), resembles an image, image quality metrics assess pixel-level differences and structural information (Wang et al., 2004), capturing directional patterns. Therefore, the evaluation involves four widely adopted metrics: MSE↓, Learned Perceptual Image Patch Similarity (LPIPS↓), Peak Signal-to-Noise Ratio (PSNR↑), and Structural Similarity Index Measure (SSIM↑).

Citation

@misc{yang2025gwrfrf,
  title={Generalizable Radio-Frequency Radiance Fields for Spatial Spectrum Synthesis},
  author={Kang Yang, Yuning Chen, Wan Du (2025)},
  year={2025},
  note={arXiv:2502.05708}
}

• arXiv: 2502.05708