Progressftx Eval

progressftx-eval

Progressive Feature Transmission for Split Inference at the Wireless Edge — Lan et al. (2021) (arXiv:2112.07244, 2021)

What this evaluates

Evaluates a progressive feature transmission protocol for split inference at the wireless edge, measuring how efficiently features are transmitted to meet target inference accuracy or uncertainty thresholds under varying channel conditions.

Datasets

• GM dataset — total ?; splits: train (-1), test (-1)
• MNIST — total ?; splits: train (-1), test (-1); HF mnist

Metrics

• average communication latency (primary) — range: slots
  • The average number of transmission slots required for a model to meet a predefined target on inference accuracy or expected uncertainty across the test set.
• inference accuracy — range: [0, 1]
  • The proportion of correctly classified samples in the test set.
• inference uncertainty — range: other
  • A scalar uncertainty estimate predicted by an auxiliary network, measured via test mean-square error during training and used as a stopping criterion.

Input / output format

Input: Partial feature maps or feature dimensions from a split inference model (LeNet or linear classifier), quantized at 64 bits per feature, transmitted over a Gaussian or fading channel with a fixed slot duration of 10 ms.

Output: Classification prediction, uncertainty estimate, and the cumulative number of transmission slots used to reach the target accuracy or uncertainty threshold.

Scoring recipe

def compute_avg_latency(dataset, target_acc, target_unc, model, unc_predictor):
    total_slots = 0
    for x, y in dataset:
        slots = 0
        while slots < 5:  # horizon K=5
            transmit_next_feature(x, slots)
            slots += 1
            if model.predict(x) >= target_acc or unc_predictor.predict(x) <= target_unc:
                break
        total_slots += slots
    return total_slots / len(dataset)

Common pitfalls

• Confusing the fixed per-slot transmission rate (Y0) with the total number of features transmitted; the protocol stops early based on feedback, so total transmitted features vary per sample.
• Overlooking that the uncertainty predictor is a separate auxiliary network trained for 50 epochs, not part of the main classifier, and its MSE is used to validate its reliability before evaluation.
• Assuming benchmark schemes use ACK/NACK feedback; one-shot compression and random-feature optimal stopping lack this feedback, leading to over-transmission and higher latency.

Evidence (verbatim from paper)

Define the average communication latency of a transmission scheme for edge inference as the average number of transmission slots required for meeting a requirement on inference accuracy or expected uncertainty.

Citation

@misc{lan2021progressftx,
  title={Progressive Feature Transmission for Split Inference at the Wireless Edge},
  author={Lan et al. (2021)},
  year={2021},
  note={arXiv:2112.07244}
}

• arXiv: 2112.07244