Token Embedding Inversion Accuracy

token-embedding-inversion-accuracy

Natural Language Understanding with Privacy-Preserving BERT — Chen Qu et al. (2021) (arXiv:2104.07504, 2021)

What this evaluates

Evaluates the privacy leakage of a token-level perturbation mechanism by measuring how easily an adversary can recover original tokens from their privatized embeddings. It probes the robustness of the privacy-preserving noise injection against nearest-neighbor-based inversion attacks.

Datasets

• SST — total ?; splits: validation (-1)
• QQP — total ?; splits: validation (-1)

Metrics

• token embedding inversion accuracy (primary) — range: [0, 1]
  • Fraction of perturbed tokens correctly recovered by finding the nearest neighbor in the BERT embedding space.
• N_w — range: [0, 1]
  • Probability of an input token not being modified by the perturbation mechanism, estimated as the fraction of 1,000 simulations where the output token matches the input.
• S_w — range: other
  • Effective support of the output distribution, estimated as the number of unique output tokens across 1,000 simulations.

Input / output format

Input: Perturbed token embeddings generated by the privacy mechanism

Output: Predicted original token

Scoring recipe

def compute_inversion_accuracy(perturbed_embs, original_tokens, vocab_embs):
    correct = 0
    for pert_emb, orig_tok in zip(perturbed_embs, original_tokens):
        pred_tok = argmin([cosine_dist(pert_emb, v_emb) for v_emb in vocab_embs])
        if pred_tok == orig_tok:
            correct += 1
    return correct / len(original_tokens)

def compute_plausible_deniability(token, vocab, n_sim=1000):
    outputs = [perturb(token) for _ in range(n_sim)]
    N_w = sum(1 for o in outputs if o == token) / n_sim
    S_w = len(set(outputs))
    return N_w, S_w

Common pitfalls

• Measures privacy leakage, not downstream NLU task utility or model accuracy.
• Relies solely on nearest neighbor search; does not evaluate more advanced ML inversion attacks.
• Plausible deniability statistics (N_w, S_w) are estimated via Monte Carlo simulation (1,000 perturbations per token) rather than empirical dataset frequencies.

Evidence (verbatim from paper)

We define an adversarial task, token embedding inversion, as recovering the original tokens based on perturbed token embeddings. We leverage nearest neighbor search for this task. Given a perturbed token embedding, we find the nearest neighbor of this embedding in the embedding space as the predicted original token. The performance is measured by accuracy.

Citation

@misc{chen2021privacybert,
  title={Natural Language Understanding with Privacy-Preserving BERT},
  author={Chen Qu et al. (2021)},
  year={2021},
  note={arXiv:2104.07504}
}

• arXiv: 2104.07504