Dinov2 Linear Eval

dinov2-linear-eval

DINOv2: Learning Robust Visual Features without Supervision — Maxime Oquab et al. (2023) (arXiv:2304.07193, 2023)

What this evaluates

Evaluates the quality of frozen self-supervised visual features by training a simple linear classifier on top of them across diverse image and video understanding tasks, probing generalization, robustness, and instance-level recognition capabilities.

Datasets

• ImageNet-1k — total 1281167; splits: train (1281167), val (50000); HF imagenet-1k
• ImageNet-V2 — total 10000; splits: test (10000); HF imagenet-v2
• ImageNet-ReaL — total 10000; splits: test (10000)
• iNaturalist — total ?; splits: val (-1); HF iNaturalist
• Places205 — total ?; splits: test (-1); HF places205
• UCF-101 — total 13320; splits: test (3780); HF ucf101
• Kinetics-400 — total ?; splits: test (-1); HF kinetics400
• Something-Something v2 — total ?; splits: test (-1); HF something-something-v2
• Oxford/Paris — total ?; splits: test (-1)
• ImageNet-A — total 7500; splits: test (7500); HF imagenet-a
• ImageNet-R — total 30000; splits: test (30000); HF imagenet-r
• ImageNet-C — total 30000; splits: test (30000); HF imagenet-c
• Sketch — total 10000; splits: test (10000); HF sketch

Metrics

• Top-1 accuracy (linear evaluation) (primary) — range: percent
  • Percentage of correctly classified instances when training a linear classifier (logistic regression or linear layer) on frozen backbone features. Computed on the validation or test set.
• kNN accuracy — range: percent
  • Accuracy computed using a k-Nearest Neighbors classifier on frozen features, typically evaluated on the validation set.
• mAP — range: percent
  • Mean Average Precision used for image retrieval benchmarks (e.g., Oxford/Paris).

Input / output format

Input: Images (and videos sampled as evenly spaced frames) fed into a frozen Vision Transformer backbone to extract feature embeddings.

Output: Class predictions from a linear probe trained on frozen features, or feature vectors for retrieval tasks.

Scoring recipe

def linear_eval(frozen_features, labels):
    # frozen_features: shape (N, D)
    # labels: shape (N,)
    model = LinearClassifier()
    model.train(frozen_features, labels)
    preds = model.predict(frozen_features)
    return accuracy(preds, labels) * 100

Common pitfalls

• Using full end-to-end fine-tuning instead of the specified linear probing protocol, which invalidates the comparison to other SSL methods.
• Not using frozen backbone weights; the evaluation explicitly requires features to remain frozen during the downstream task.
• Confusing linear evaluation with kNN evaluation; the paper reports both but uses linear evaluation as the primary headline metric.

Evidence (verbatim from paper)

We evaluate the quality of features by training a simple classifier over a frozen backbone, and do not perform finetuning of the backbone weights. Following previous work, we use a linear model for simplicity, ensuring a reproducible evaluation... We report Top-1 accuracy on the validation set for publicly available models trained on public or private data, and with or without text supervision.

Citation

@misc{oquab2023dinov2,
  title={DINOv2: Learning Robust Visual Features without Supervision},
  author={Maxime Oquab et al. (2023)},
  year={2023},
  note={arXiv:2304.07193}
}

• arXiv: 2304.07193