Mosquitofusion Eval

mosquitofusion-eval

MosquitoFusion: A Multiclass Dataset for Real-Time Detection of Mosquitoes, Swarms, and Breeding Sites Using Deep Learning — Sayeedi et al. (2024) (arXiv:2404.01501, 2024)

What this evaluates

Evaluates real-time multiclass object detection capabilities for identifying individual mosquitoes, mosquito swarms, and breeding sites in natural environments. It measures how well a model can localize and classify these distinct biological and environmental targets under varying real-world conditions.

Datasets

• MosquitoFusion — total 1204; splits: train (-1), val (-1), test (-1); repo https://github.com/faiyazabdullah/MosquitoFusion

Metrics

• mAP@50 (primary) — range: percent
  • Mean Average Precision at an Intersection over Union (IoU) threshold of 0.50. Calculated as the mean of the Average Precision scores across all three classes (mosquito, swarm, breeding site).
• Precision — range: percent
  • The ratio of true positive detections to the total number of detections (true positives + false positives).
• Recall — range: percent
  • The ratio of true positive detections to the total number of ground truth instances (true positives + false negatives).

Input / output format

Input: RGB images captured in real-world environments containing mosquitoes, swarms, or breeding sites.

Output: Bounding box coordinates and class labels for each detected instance.

Scoring recipe

def compute_metrics(preds, gold, iou_thresh=0.5):
    tp, fp, fn = 0, 0, 0
    for p_box, p_cls in preds:
        matched = False
        for g_box, g_cls in gold:
            if p_cls == g_cls and iou(p_box, g_box) >= iou_thresh:
                tp += 1
                matched = True
                break
        if not matched:
            fp += 1
    fn = len(gold)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    return precision, recall

Common pitfalls

• The paper does not disclose the exact number of images in the train, validation, and test splits.
• The model's performance degrades when distinguishing mosquito swarms from swarms of other insects, a limitation explicitly acknowledged by the authors.
• Heavy reliance on real-world image augmentations means results may not generalize to synthetic or highly controlled datasets.

Evidence (verbatim from paper)

validated with YOLOv8s achieving 57.1% mAP@50, 73.4% precision, and 50.5% recall. The split into training, validation, and test sets ensures reliable evaluation, emphasizing the dataset's value for training effective mosquito detection model.

Citation

@misc{sayeedi2024mosquitofusion,
  title={MosquitoFusion: A Multiclass Dataset for Real-Time Detection of Mosquitoes, Swarms, and Breeding Sites Using Deep Learning},
  author={Sayeedi et al. (2024)},
  year={2024},
  note={arXiv:2404.01501}
}

• arXiv: 2404.01501