# Securerouter Eval

> This evaluation probes the accuracy and inference efficiency of an encrypted routing framework for secure Transformer inference. It measures how well a cost-aware router dynamically selects smaller MPC-optimized models from a pool to balance privacy-preserving computation costs with task-specific accuracy requirements. Use when the user wants to benchmark on GLUE, or asks about evaluating this task. Reports Inference Speed-up.

- Skill: `qhjqhj00/securerouter-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/securerouter-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/securerouter-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/securerouter-eval

---


# securerouter-eval

> SecureRouter: Encrypted Routing for Efficient Secure Inference — Zhang et al. (2026) (arXiv:2604.15499, 2026)

## What this evaluates

This evaluation probes the accuracy and inference efficiency of an encrypted routing framework for secure Transformer inference. It measures how well a cost-aware router dynamically selects smaller MPC-optimized models from a pool to balance privacy-preserving computation costs with task-specific accuracy requirements.

## Datasets

- **GLUE** — total ?; splits: test (-1); HF `glue`

## Metrics

- `Accuracy` — range: [0, 1]
  - Percentage of correctly predicted labels on classification tasks (MNLI, RTE, SST-2, QNLI).
- `F1 score` — range: [0, 1]
  - Harmonic mean of precision and recall for binary/multi-class tasks (MRPC, QQP).
- `Matthews correlation coefficient` — range: [-1, 1]
  - Correlation coefficient between true and predicted labels, robust to class imbalance (CoLA).
- `Pearson/Spearman correlation` — range: [-1, 1]
  - Average of Pearson and Spearman rank correlations for regression tasks (STS-B).
- `Inference Speed-up` **(primary)** — range: other
  - Ratio of baseline BERT-Large MPC latency to the weighted sum of selected expert latencies plus router overhead: (N × C_Large) / Σ(C_selected + C_router).

## Input / output format

**Input**: Encrypted text sequences representing GLUE benchmark tasks (classification or regression).

**Output**: Predicted class labels or regression scores, along with a routing decision indicating which expert model (Tiny, Base, or Large) processes the sample.

## Scoring recipe

```python
For each task, compute the designated metric (accuracy, F1, MCC, or avg correlation) against gold labels.
Compute speed-up as: speedup = (N * C_Large) / sum(C_selected[i] + C_router for i in range(N)).
Report per-task metrics and average speed-up across tasks.
```

## Common pitfalls

- Speed-up values are projected based on router profiling and simulated 2PC environments, not real-world distributed MPC deployments.
- Accuracy drops on syntactically strict tasks (e.g., CoLA) when smaller experts are selected, highlighting a capacity-accuracy trade-off.
- MPC latency is highly sensitive to network bandwidth (10 Gbps simulated) and hardware configuration, making cross-environment comparisons difficult.

## Evidence (verbatim from paper)

> Specifically, performance is measured using accuracy (for MNLI, RTE, SST-2, and QNLI), the F1 score (for MRPC and QQP), the Matthews correlation coefficient (for CoLA), and the average of Pearson and Spearman correlations (for STS-B).

## Citation

```bibtex
@misc{zhang2026securerouter,
  title={SecureRouter: Encrypted Routing for Efficient Secure Inference},
  author={Zhang et al. (2026)},
  year={2026},
  note={arXiv:2604.15499}
}
```

- arXiv: 2604.15499