Ocean Workbench Eval

ocean-workbench-eval

OceanSAR-2: A Universal Feature Extractor for SAR Ocean Observation — Tuel et al. (2026) (arXiv:2601.07392, 2026)

What this evaluates

Evaluates SAR foundation models on a suite of ocean observation tasks, including geophysical pattern classification, continuous regression for wave height and wind parameters, and iceberg object detection. It tests both zero-shot feature transferability and fine-tuning adaptability across diverse geophysical benchmarks.

Datasets

• Ocean Workbench — total ?; splits: test (-1)

Metrics

• TenGeoP accuracy (primary) — range: percent
  • Percentage of correctly classified geophysical patterns. Computed as correct predictions divided by total samples, reported in percent.
• SWH regression error — range: other
  • Regression error for significant wave height prediction. Lower values indicate better performance, reported in meters.
• Wspd regression error — range: other
  • Regression error for wind speed estimation. Lower values indicate better performance, reported in meters per second.
• Wdir regression error — range: other
  • Regression error for wind direction estimation. Lower values indicate better performance, reported in degrees.
• F1-score — range: [0, 1]
  • Harmonic mean of precision and recall for iceberg detection, computed at an IoU threshold of 0.1 and a confidence score threshold of 0.5.

Input / output format

Input: Sentinel-1 Wave Mode SAR imagery with calibrated σ⁰ backscatter. Models receive raw image patches or full scenes depending on the backbone architecture.

Output: Zero-shot: image-level embeddings (384-2048 dimensions). Fine-tuning: class labels for TenGeoP, scalar regression values for SWH/Wspd/Wdir, and bounding boxes with confidence scores for iceberg detection.

Scoring recipe

def evaluate(predictions, gold, mode='zero-shot', task=None):
    if mode == 'zero-shot':
        return kNN_classify_or_regress(predictions, gold, k=5)
    elif mode == 'fine-tune':
        if task == 'TenGeoP':
            return accuracy(predictions, gold)
        elif task in ['SWH', 'Wspd', 'Wdir']:
            return rmse(gold, predictions)
        elif task == 'YOLOIB':
            return compute_f1(predictions, gold, iou_thresh=0.1, score_thresh=0.5)

Common pitfalls

• TerraMind outputs feature maps rather than image-level embeddings; global max averaging must be applied to the final feature map to match reported performance.
• Zero-shot kNN evaluation is explicitly excluded for wind direction and iceberg detection benchmarks due to architectural incompatibility.
• Fine-tuning stability varies significantly across models; some baselines (e.g., TerraMind on YOLOIB) fail to converge or lack available code, yielding unreliable results marked with asterisks.

Evidence (verbatim from paper)

We record the F1-score on predicted boxes, using an IoU threshold of 0.1 and score threshold of 0.5. In zero-shot mode, we apply kNN classification and regression to the image features directly produced by the models (excluding the wind direction and iceberg benchmarks, for which this is not possible).

Citation

@misc{tuel2026oceansar2,
  title={OceanSAR-2: A Universal Feature Extractor for SAR Ocean Observation},
  author={Tuel et al. (2026)},
  year={2026},
  note={arXiv:2601.07392}
}

• arXiv: 2601.07392