Bscd Fsl Eval

bscd-fsl-eval

A Broader Study of Cross-Domain Few-Shot Learning — Guo et al. (2019) (arXiv:1912.07200, 2019)

What this evaluates

Evaluates cross-domain few-shot learning generalization from a source domain (ImageNet) to specialized imaging domains (agriculture, satellite, dermatology, radiology) that vary in perspective distortion, semantic content, and color depth.

Datasets

• CropDiseases — total ?; splits: train (-1), test (-1)
• EuroSAT — total ?; splits: train (-1), test (-1)
• ISIC2018 — total ?; splits: train (-1), test (-1)
• ChestX — total ?; splits: train (-1), test (-1)

Metrics

• accuracy (primary) — range: [0, 1]
  • Standard classification accuracy: the proportion of correctly predicted query images out of the total query set across all few-shot episodes.

Input / output format

Input: Images from a source domain (ImageNet) for meta-training, and few-shot support/query sets from target domains (CropDiseases, EuroSAT, ISIC2018, ChestX) for evaluation.

Output: Class labels for query images in each few-shot episode.

Scoring recipe

correct = 0
total = 0
for episode in episodes:
    predictions = model.predict(episode.query_images, episode.support_images, episode.support_labels)
    correct += sum(p == g for p, g in zip(predictions, episode.query_labels))
    total += len(episode.query_labels)
return correct / total

Common pitfalls

• Datasets span a wide spectrum of image characteristics (perspective, semantics, color depth), so performance drops may reflect domain shift rather than algorithmic failure.
• The benchmark introduction does not specify few-shot episode construction (e.g., N-way K-shot splits), making exact reproduction difficult without the full paper or codebase.

Evidence (verbatim from paper)

The selected datasets reflect well-curated real-world use cases for few-shot learning. ... Having a few-shot learning model trained on a source domain such as ImageNet that can generalize to domains such as these, is highly desirable, as it enables effective learning for rare categories in new types of images, which has previously not been studied in detail. Evaluation is typically conducted using standard accuracy on query sets.

Citation

@misc{guo2019broader,
  title={A Broader Study of Cross-Domain Few-Shot Learning},
  author={Guo et al. (2019)},
  year={2019},
  note={arXiv:1912.07200}
}

• arXiv: 1912.07200