Imagenet Multiplexing Eval

imagenet-multiplexing-eval

Runtime Deep Model Multiplexing for Reduced Latency and Energy Consumption Inference — Eshratifar et al. (2020) (arXiv:2001.05870, 2020)

What this evaluates

Evaluates the trade-off between inference latency, energy consumption, and classification accuracy when dynamically routing image inputs between a lightweight mobile model and a powerful cloud model using a learned neural multiplexer.

Datasets

• ImageNet ILSVRC 2012 — total ?; splits: train (-1), val (-1)

Metrics

• accuracy (primary) — range: percent
  • Top-1 classification accuracy: the fraction of input images correctly classified by the selected model (mobile or cloud).
• latency — range: other
  • End-to-end inference time measured in milliseconds (ms), including computation and communication overheads.
• energy_consumption — range: other
  • Energy consumed per inference measured in millijoules (mJ) using an INA226 power sensor on the mobile device.

Input / output format

Input: RGB image passed to a 4-layer CNN multiplexer, which outputs a continuous routing score between 0 and 1.

Output: Binarized routing decision (0 = local/mobile, 1 = cloud) based on a 0.5 threshold, followed by the class label predicted by the selected model.

Scoring recipe

def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)

# Latency and energy are measured via system timers and INA226 sensor respectively,
# not computed from (predictions, gold). They are averaged over the test set.

Common pitfalls

• Directly comparing hybrid latency/energy to mobile-only baselines without accounting for the multiplexer overhead and communication costs, which the authors explicitly note is an unfair comparison since the hybrid approach prioritizes accuracy gains.
• Assuming the multiplexer threshold is universally fixed at 0.5; the paper demonstrates that sweeping thresholds yields different optimal values (e.g., 0.288 for cloud-based API inference).
• Overlooking that the reported accuracy improvements stem primarily from offloading hard examples to the cloud, rather than the multiplexer itself improving classification capability.

Evidence (verbatim from paper)

Table I: The latency, percentage of local inference, and accuracy of mobile-only, cloud-only and hybrid (multiplexing) methods.

Citation

@misc{eshratifar2020multiplexing,
  title={Runtime Deep Model Multiplexing for Reduced Latency and Energy Consumption Inference},
  author={Eshratifar et al. (2020)},
  year={2020},
  note={arXiv:2001.05870}
}

• arXiv: 2001.05870