qhjqhj00 saga-3d-mesh-attack-eval
SAGA: Spectral Adversarial Geometric Attack on 3D Meshes — Stolik et al. (2022) (arXiv:2211.13775, 2022)
What this evaluates
Evaluates the effectiveness of a spectral geometric adversarial attack on 3D mesh autoencoders by measuring how well perturbed meshes deceive a downstream classifier and evade detection.
Datasets
- CoMA — total 10649; splits: train (8325), val (926), test (1398)
- SMAL — total 9918; splits: train (-1), val (-1), test (-1)
Metrics
Targeted classification accuracy(primary) — range: percent- Percentage of adversarial reconstructions that a PointNet classifier correctly labels as the target class. Calculated as (correct target predictions / total attacked pairs) * 100.
Untargeted classification accuracy— range: percent- Percentage of adversarial reconstructions that the classifier labels as any class different from the source class. Calculated as (non-source predictions / total attacked pairs) * 100.
Detection accuracy— range: percent- Percentage of adversarial meshes correctly identified as adversarial by a trained detector network. Lower values indicate a more successful, imperceptible attack.
Input / output format
Input: 3D mesh vertex coordinates and connectivity (source shape)
Output: Adversarial 3D mesh (perturbed source shape) fed into a Mesh Autoencoder, followed by classification/detection of the AE's reconstruction.
Scoring recipe
def compute_metrics(reconstructions, source_classes, target_classes, detector):
targeted_correct = sum(1 for r, t in zip(reconstructions, target_classes) if classifier.predict(r) == t)
targeted_acc = (targeted_correct / len(reconstructions)) * 100
untargeted_correct = sum(1 for r, s in zip(reconstructions, source_classes) if classifier.predict(r) != s)
untargeted_acc = (untargeted_correct / len(reconstructions)) * 100
detected_correct = sum(1 for r in reconstructions if detector.predict(r) == 'adversarial')
detection_acc = (detected_correct / len(reconstructions)) * 100
return targeted_acc, untargeted_acc, detection_acc
Common pitfalls
- Lower detection accuracy indicates a more successful attack, which is the inverse of standard classification metrics.
- Metrics are computed on the Mesh Autoencoder's reconstruction of the adversarial mesh, not directly on the adversarial mesh itself.
- The attack is optimized per source-target pair (50 sources per class × 10 targets = 5500 pairs for CoMA), not a universal attack.
Evidence (verbatim from paper)
Table 1 shows the accuracy obtained from classifying the adversarial reconstructions as the target, in the targeted case, or differently from the source, in the untargeted case. The experiment included all the attacked pairs. We compare our attack with Lang et al.’s PC attack and with the clean target reconstructions.
Citation
@misc{stolik2022saga,
title={SAGA: Spectral Adversarial Geometric Attack on 3D Meshes},
author={Stolik et al. (2022)},
year={2022},
note={arXiv:2211.13775}
}
- arXiv: 2211.13775