Jointdnn Benchmarks Eval

jointdnn-benchmarks-eval

JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services — Eshratifar et al. (2018) (arXiv:1801.08618, 2018)

What this evaluates

Evaluates a layer-granular DNN offloading framework by measuring inference latency and energy consumption across standard discriminative, generative, and autoencoder neural network architectures on mobile-cloud setups.

Datasets

• JointDNN Deep Architecture Benchmarks (AlexNet, OverFeat, VGG16, Deep Speech, ResNet, NiN, Chair, Pix2Pix) — total ?; splits: test (-1)

Metrics

• latency (primary) — range: other
  • Total end-to-end inference time from input submission to output retrieval, measured in seconds. Includes computation time on mobile/cloud and communication transmission time.
• energy_consumption — range: other
  • Total energy expended by the mobile device during offloading, calculated from communication power (linear model based on throughput) and computation power, measured in millijoules or watt-seconds.

Input / output format

Input: DNN model specifications (architecture type, layer count) and mobile network parameters (3G/4G/Wi-Fi download/upload speeds and power coefficients).

Output: Optimal offloading partition points across DNN layers, along with measured latency and energy consumption values for each configuration.

Scoring recipe

def evaluate_offloading(model, network_params):
    total_latency = 0
    total_energy = 0
    for layer in model.layers:
        comp_time, comp_energy = compute_layer(layer, device)
        comm_time, comm_energy = transmit(layer.output_size, network_params)
        total_latency += comp_time + comm_time
        total_energy += comp_energy + comm_energy
    return total_latency, total_energy

Common pitfalls

• Assumes constant network throughput; actual mobile networks experience significant variance and congestion.
• Uses idealized linear power models for communication that may not capture hardware-specific idle states or protocol overheads.
• Benchmarks are restricted to specific model families, potentially overlooking performance on modern transformer or diffusion architectures.

Evidence (verbatim from paper)

we have chosen three common application types of DNNs as shown in Table II: 1. Discriminative neural networks... 2. Generative neural networks... 3. Autoencoders... The method leverages layer-specific output size characteristics... to optimize offloading decisions, and achieves up to 32× reduction in mobile energy and 18× reduction in latency compared to cloud-only or mobile-only approaches.

Citation

@misc{eshratifar2018jointdnn,
  title={JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services},
  author={Eshratifar et al. (2018)},
  year={2018},
  note={arXiv:1801.08618}
}

• arXiv: 1801.08618