Text Sanitization Reconstruction Eval

text-sanitization-reconstruction-eval

On the Vulnerability of Text Sanitization — Tong et al. (2024) (arXiv:2410.17052, 2024)

What this evaluates

Evaluates the vulnerability of differential privacy-based text sanitization methods by measuring how accurately an attacker can reconstruct original sensitive or personally identifiable information (PII) tokens from their sanitized counterparts. It probes the effectiveness of Bayesian inference-based reconstruction attacks against state-of-the-art sanitization defenses.

Datasets

• SST-2 — total 20000; splits: private (20000), shadow (1000)
• AGNEWS — total 20000; splits: private (20000), shadow (1000)
• QNLI — total 20000; splits: private (20000), shadow (1000)
• Yelp — total 20000; splits: private (20000), shadow (1000)

Metrics

• ASR (primary) — range: [0, 1]
  • Attack Success Rate is calculated as the ratio of correctly reconstructed sensitive tokens to the total number of sensitive tokens in the private dataset.

Input / output format

Input: Sanitized text sequences (with DP-perturbed sensitive tokens) and a disjoint shadow dataset for estimating token distributions.

Output: Reconstructed sensitive tokens corresponding to the perturbed positions in the sanitized text.

Scoring recipe

def compute_asr(predictions, gold_tokens):
    correct = sum(1 for p, g in zip(predictions, gold_tokens) if p == g)
    return correct / len(gold_tokens)

Common pitfalls

• Shadow dataset must be strictly disjoint from the private reconstruction dataset to avoid data leakage and inflated ASR.
• The parameter K (top-k candidates) significantly impacts contextual attack performance, plateauing around K=10.
• Privacy budget (epsilon) heavily influences ASR; lower epsilon drastically reduces reconstruction success and requires careful reporting.

Evidence (verbatim from paper)

The ASR of both the Context-free Bayesian Attack and the Contextual Bayesian Attack exceeded that of the baseline attacks in most cases.

Citation

@misc{tong2024vulnerability,
  title={On the Vulnerability of Text Sanitization},
  author={Tong et al. (2024)},
  year={2024},
  note={arXiv:2410.17052}
}

• arXiv: 2410.17052