Yoloe Lvis Coco Eval

yoloe-lvis-coco-eval

YOLOE: Real-Time Seeing Anything — Wang et al. (2025) (arXiv:2503.07465, 2025)

What this evaluates

Evaluates open-vocabulary object detection and segmentation capabilities using text, visual, and prompt-free inputs on zero-shot and fine-tuned settings.

Datasets

• LVIS — total ?; splits: minival (-1), val (-1)
• COCO — total ?; splits: train (-1), val (-1)

Metrics

• Fixed AP (primary) — range: [0, 100]
  • Average precision computed with a fixed number of detections per image (typically 100), evaluated in a zero-shot manner on the LVIS minival subset.
• APm — range: [0, 100]
  • Mean average precision for segmentation masks, evaluated on the LVIS val set.
• AP — range: [0, 100]
  • Standard average precision for detection and segmentation, evaluated on COCO after fine-tuning.
• FPS — range: other
  • Frames per second measured on Nvidia T4 GPU with TensorRT and iPhone 12 with CoreML.

Input / output format

Input: RGB image; optional text prompt (category names), visual prompt (bounding box crops), or no prompt (prompt-free).

Output: List of predicted bounding boxes and/or segmentation masks with associated confidence scores and category labels.

Scoring recipe

def compute_fixed_ap(predictions, ground_truth):
    # predictions: list of dicts with 'bbox', 'score', 'category'
    # ground_truth: list of dicts with 'bbox', 'category'
    # 1. Sort predictions by score descending.
    # 2. Keep top 100 detections per image.
    # 3. Match to ground truth using IoU >= 0.5.
    # 4. Compute precision-recall curve and average.
    return ap_score

Common pitfalls

• Fixed AP on LVIS minival uses a fixed number of detections per image, unlike standard AP which uses all detections.
• Prompt-free evaluation maps open-ended predictions to a built-in vocabulary of 4585 categories using a text encoder, rather than the LVIS category list.
• Visual prompt evaluation averages embeddings from N=16 randomly sampled training images per category, not test images.

Evidence (verbatim from paper)

By default, Fixed AP [[7]] on LVIS minival subset is reported. For transferring to COCO, standard AP is evaluated, following [[1], [21]].

Citation

@misc{wang2025yoloe,
  title={YOLOE: Real-Time Seeing Anything},
  author={Wang et al. (2025)},
  year={2025},
  note={arXiv:2503.07465}
}

• arXiv: 2503.07465