Roughness Index Distance

roughness-index-distance

Roughness Index and Roughness Distance for Benchmarking Medical Segmentation — Rathour et al. (2021) (arXiv:2103.12350, 2021)

What this evaluates

Evaluates surface irregularities and topological consistency between predicted and ground-truth 3D medical segmentation masks. It quantifies local surface roughness, relative roughness differences, and average surface distance to detect spikes, holes, and smoothing artifacts.

Datasets

• (no dataset; pure metric skill)

Metrics

• Roughness Index (RI) (primary) — range: [0, ∞)
  • RI = (1/M) * Σ_{w=1 to M} [ (1/N) * Σ_{i=1 to N} |ζ_i - Mean(ζ_window)| ], where ζ is the L2 distance from a surface point to the center of gravity, and M is the number of fixed-size surface windows.
• Roughness Ratio (RR) — range: [0, ∞)
  • RR = |RI_P - RI_G| / RI_G, measuring the relative roughness difference between predicted and ground-truth segmentations.
• Average Roughness Distance (ARD) — range: [0, ∞)
  • ARD = Mean(|ζ̂_m|), where ζ̂_m is the element-wise difference between the predicted and ground-truth distance matrices from the center of gravity.

Input / output format

Input: Two 3D binary segmentation masks: predicted (P) and ground-truth (G), represented as voxel grids.

Output: Scalar values for Roughness Index (RI), Roughness Ratio (RR), and Average Roughness Distance (ARD).

Scoring recipe

def compute_roughness_metrics(P, G):
    # 1. Compute center of gravity C0 for the surface
    C0 = mean([v for v in P if v in surface])
    # 2. Compute distance matrices from C0 for P and G
    zeta_P = array([L2(v, C0) for v in P_surface])
    zeta_G = array([L2(v, C0) for v in G_surface])
    # 3. Compute RI for P and G using fixed windows
    RI_P = mean([mean([abs(d - mean(window_d)) for d in window_d]) for window in windows])
    RI_G = mean([mean([abs(d - mean(window_d)) for d in window_d]) for window in windows])
    # 4. Compute ARD
    zeta_hat = zeta_P - zeta_G
    ARD = mean(abs(zeta_hat))
    # 5. Compute RR
    RR = abs(RI_P - RI_G) / RI_G
    return RI_P, RI_G, RR, ARD

Common pitfalls

• Using a fixed laser-plane height coordinate instead of the center-of-gravity distance, which breaks the closed-contour assumption for medical masks.
• Failing to normalize RI by the number of windows M, causing the metric to scale with surface resolution or windowing strategy.
• Division by zero in RR when the ground-truth segmentation is perfectly smooth (RI_G = 0).

Evidence (verbatim from paper)

The Roughness Index (RI) in 3D can be calculated by dividing the segmentation surface S into small surface element ∂S^w of a fixed window size w, and then calculating the average deviation of ζ from the mean ζ_Mean for all surface voxels in the surface element ∂S^w as illustrated in Fig:[9(b)]... We also propose Average Roughness Distance (ARD) which as the name suggest is the average surface/roughness distance between two objects as shown in Eq:[23]... ARD = Mean(|ζ̂_m|)

Citation

@misc{rathour2021roughness,
  title={Roughness Index and Roughness Distance for Benchmarking Medical Segmentation},
  author={Rathour et al. (2021)},
  year={2021},
  note={arXiv:2103.12350}
}

• arXiv: 2103.12350