Domainnet Eval

domainnet-eval

Moment Matching for Multi-Source Domain Adaptation — Peng et al. (2018) (arXiv:1812.01754, 2018)

What this evaluates

Evaluates multi-source domain adaptation methods on image classification tasks across multiple domains with varying visual styles and categories.

Datasets

• DomainNet — total 600000; splits: train (-1), test (-1)

Metrics

• average accuracy (primary) — range: percent
  • Percentage of correctly classified images in the target domain test set, averaged across all source-to-target domain transfer combinations.

Input / output format

Input: RGB images from multiple source domains and a target domain, with category labels.

Output: Predicted class labels for each image in the target domain test set.

Scoring recipe

correct = 0
total = 0
for src, tgt in source_to_target_pairs:
    preds = model.predict(test_imgs[tgt])
    correct += sum(p == g for p, g in zip(preds, gold[tgt]))
    total += len(gold[tgt])
return (correct / total) * 100

Common pitfalls

• Negative transfer can degrade performance on specific domain pairs (e.g., MNIST-M).
• The 70/30 train/test split is applied independently per domain, not globally across the dataset.
• Performance is averaged over all source-to-target transfer directions, masking domain-specific failures.

Evidence (verbatim from paper)

Our model M³SDA achieves an 86.13% average accuracy, and M³SDA-β boosts the performance to 87.65%, outperforming other baselines by a large margin.

Citation

@misc{peng2018momentmatching,
  title={Moment Matching for Multi-Source Domain Adaptation},
  author={Peng et al. (2018)},
  year={2018},
  note={arXiv:1812.01754}
}

• arXiv: 1812.01754