qhjqhj00 braTS-pLGG-classification-eval
Generating 3D Brain Tumor Regions in MRI using Vector-Quantization Generative Adversarial Networks — Meng Zhou et al. (arXiv:2310.01251, 2023)
What this evaluates
Evaluates the effectiveness of synthetic 3D MRI tumor ROI generation for data augmentation by measuring downstream binary classification performance on imbalanced brain tumor subtypes.
Datasets
- BraTS 2019 — total 335; splits: train (285), test (50)
- SickKids pLGG — total 214; splits: train (174), test (40)
Metrics
AUC(primary) — range: [0, 1]- Area Under the Receiver Operating Characteristic Curve, measuring the model's ability to distinguish between classes across all classification thresholds.
Accuracy— range: [0, 1]- Proportion of correctly classified instances out of the total test set instances.
Input / output format
Input: 3D MRI tumor ROI volumes of size 128×128×128, normalized to [-1, 1].
Output: Binary classification label (HGG/LGG or BRAF Fusion/V600E).
Scoring recipe
def compute_metrics(predictions, labels):
acc = np.mean(predictions == labels)
auc = roc_auc_score(labels, predictions)
return {'accuracy': acc, 'AUC': auc}
Common pitfalls
- Test set is artificially balanced (25/25 or 20/20 per class) despite imbalanced training data, which may overestimate minority-class performance compared to real-world clinical distributions.
- Evaluation relies entirely on downstream classification metrics (AUC/Accuracy) rather than direct image quality measures (e.g., FID, SSIM), making it difficult to isolate generation fidelity from classifier robustness.
Evidence (verbatim from paper)
We randomly hold out 25 patients from both HGG and LGG as a standalone test set; these data are hidden from either training the 3D-VQGAN or the classifier. ... We use the 3D ResNet-50 model and trained for 50 epochs for all classification experiments. AUC and accuracy are computed on this test set.
Citation
@misc{zhou2023generating3d,
title={Generating 3D Brain Tumor Regions in MRI using Vector-Quantization Generative Adversarial Networks},
author={Meng Zhou et al.},
year={2023},
note={arXiv:2310.01251}
}
- arXiv: 2310.01251