Orsi Sod Eval

orsi-sod-eval

ORSIFlow: Saliency-Guided Rectified Flow for Optical Remote Sensing Salient Object Detection — Haojing Chen et al. (arXiv:2603.28584, 2026)

What this evaluates

This benchmark evaluates optical remote sensing salient object detection models by measuring their ability to accurately segment prominent objects from complex, cluttered backgrounds. It probes structural consistency, boundary precision, and error magnitude across varying object scales and scene complexities.

Datasets

• ORSSD — total 800; splits: train (600), test (200)
• EORSSD — total 2000; splits: train (1400), test (600)
• ORSI-4199 — total 4199; splits: train (2000), test (2199)

Metrics

• S-measure ($S_{\alpha}$) — range: [0, 1]
  • Combines region-level and object-level similarity to evaluate structural consistency between prediction and ground truth.
• maximum F-measure ($F_{\beta}^{max}$) (primary) — range: [0, 1]
  • Computes the harmonic mean of precision and recall at the optimal threshold across a range of values, emphasizing the precision-recall tradeoff.
• maximum E-measure ($E_{\xi}^{max}$) — range: [0, 1]
  • Aligns local pixel-level and global image-level features to evaluate enhanced structural similarity.
• Mean Absolute Error (MAE) — range: [0, 1]
  • Calculates the mean of absolute pixel-wise differences between the predicted saliency map and the ground truth mask.

Input / output format

Input: Optical remote sensing images resized to 352×352 pixels.

Output: Pixel-wise saliency masks (soft or binary maps) matching the input resolution.

Scoring recipe

def compute_metrics(pred, gt):
    # pred, gt: float tensors [H, W] in [0, 1]
    mae = np.mean(np.abs(pred - gt))
    thresholds = np.linspace(0, 1, 255)
    f_scores = []
    for t in thresholds:
        p = (pred > t).astype(float)
        prec = np.sum(p * gt) / (np.sum(p) + 1e-8)
        rec = np.sum(p * gt) / (np.sum(gt) + 1e-8)
        f = (1.5 * prec * rec) / (0.5 * prec + rec + 1e-8)
        f_scores.append(f)
    f_max = max(f_scores)
    return mae, f_max

Common pitfalls

• Failing to resize inputs to the fixed 352×352 resolution used in the paper, which significantly alters metric values.
• Using a single fixed threshold for F-measure instead of computing the maximum over a range of thresholds ($F_{\beta}^{max}$).
• Not using the official open-source code and recommended settings for baseline comparisons, leading to unfair performance gaps.

Evidence (verbatim from paper)

We employed four widely used metrics to quantitatively evaluate the performance of all the methods, including S-measure ($S_{a}$), Mean Absolute Error (MAE), maximum E-measure ($E_{\xi}^{max}$), and maximum F-measure ($F_{\beta}^{max}$).

Citation

@misc{chen2026orsiflow,
  title={ORSIFlow: Saliency-Guided Rectified Flow for Optical Remote Sensing Salient Object Detection},
  author={Haojing Chen et al.},
  year={2026},
  note={arXiv:2603.28584}
}

• arXiv: 2603.28584