Leafnet Eval

leafnet-eval

LeafNet: A Large-Scale Dataset and Comprehensive Benchmark for Foundational Vision-Language Understanding of Plant Diseases — Khang Nguyen Quoc et al. (2026) (arXiv:2602.13662, 2026)

What this evaluates

Evaluates vision and vision-language models on plant disease diagnosis, including fine-grained image classification, few-shot adaptation, and zero-shot visual question answering. It probes the models' ability to recognize subtle visual symptoms, reason over taxonomic pathogen information, and generalize across agricultural domains.

Datasets

• LeafNet — total 186000; splits: train (-1), test (-1), Tiny (-1), All (-1); repo https://github.com/EnalisUs/LeafBench

Metrics

• Accuracy (primary) — range: percent
  • Percentage of correctly predicted class labels or VQA answers out of the total number of instances.
• F1-score — range: [0, 1]
  • Harmonic mean of precision and recall, computed per class and macro-averaged.
• AUC — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, measuring the model's ability to distinguish between classes.

Input / output format

Input: Single leaf image for classification tasks; image paired with a natural language question/prompt for VQA tasks.

Output: Discrete disease class label (e.g., HDC, DC, CSI, SNC, PC, SI) for classification; free-text answer for VQA.

Scoring recipe

def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels) * 100

def compute_f1(predictions, gold_labels):
    return f1_score(gold_labels, predictions, average='macro') * 100

Common pitfalls

• Linear probing on modern architectures (e.g., EfficientNetV2S) collapses to <20% accuracy, indicating poor transferability of ImageNet features to plant pathology.
• Fine-grained tasks like Species Identification (SI) and Pathogen Classification (PC) often yield near-chance performance for generic VLMs, requiring domain-specific adaptation.
• The 'Tiny' subset must be validated against the 'All' benchmark to ensure it accurately reflects difficulty distribution without sampling bias.

Evidence (verbatim from paper)

While traditional vision models remain competitive in standard classification tasks such as HDC, achieving 94.27% accuracy, they struggle to generalize to semantically demanding tasks like SNC and SI (e.g., 61.88% and 59.22% with DenseNet121 and SwinT, respectively).

Citation

@misc{nguyenquoc2026leafnet,
  title={LeafNet: A Large-Scale Dataset and Comprehensive Benchmark for Foundational Vision-Language Understanding of Plant Diseases},
  author={Khang Nguyen Quoc et al. (2026)},
  year={2026},
  note={arXiv:2602.13662}
}

• arXiv: 2602.13662