qhjqhj00 sst2-imdb-eval
Bridging Classical and Quantum Computing for Next-Generation Language Models — Pan et al. (2025) (arXiv:2508.07026, 2025)
What this evaluates
Evaluates the binary sentiment classification capability of hybrid quantum-classical language models on both short and long text sequences. It probes whether adaptive quantum routing and attention mechanisms provide measurable accuracy gains over purely classical or purely quantum baselines on standard NLP benchmarks.
Datasets
- SST-2 — total ?; splits: test (-1)
- IMDB — total ?; splits: test (-1)
Metrics
Accuracy(primary) — range: percent- Percentage of correctly classified instances out of the total number of instances in the evaluation set.
Precision— range: percent- Ratio of true positive predictions to the total number of positive predictions.
Recall— range: percent- Ratio of true positive predictions to the total number of actual positive instances.
F1-Score— range: percent- Harmonic mean of Precision and Recall.
Input / output format
Input: Raw text sequences labeled with binary sentiment (positive/negative).
Output: Predicted sentiment class label (positive or negative).
Scoring recipe
def calculate_accuracy(predictions, gold_labels):
correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
return (correct / len(gold_labels)) * 100
Common pitfalls
- Pure quantum models (e.g., VQC, Quantum LSTM) fail catastrophically on these tasks without classical integration, yielding near-random accuracy.
- Performance gains are highly dependent on sequence length and semantic complexity; quantum advantage scales exponentially with task difficulty rather than uniformly.
- Reported training/inference times are based on 20-qubit classical simulators, not real quantum hardware, which drastically underestimates future latency.
Evidence (verbatim from paper)
On SST-2, AQCF achieves 81.88% accuracy, outperforming the classical Transformer baseline by 2.52 percentage points and the standard Quantum Transformer by 2.29 percentage points. ... The performance gap widens on the more challenging IMDB dataset, where AQCF attains 86.30% accuracy—a 4.12 percentage point improvement over the classical baseline.
Citation
@misc{pan2025bridging,
title={Bridging Classical and Quantum Computing for Next-Generation Language Models},
author={Pan et al. (2025)},
year={2025},
note={arXiv:2508.07026}
}
- arXiv: 2508.07026