Dvfs Latency Energy Eval

dvfs-latency-energy-eval

DVFS-Aware DNN Inference on GPUs: Latency Modeling and Performance Analysis — Han et al. (2025) (arXiv:2502.06295, 2025)

What this evaluates

Evaluates the accuracy of a data-driven DVFS-aware latency model for DNN inference on GPUs against a traditional FLOPs-based benchmark. It probes the model's ability to predict real-world inference time and energy consumption under varying frequency settings, deadlines, and cooperative offloading scenarios.

Datasets

• CIFAR10 — total ?; splits: test (-1)

Metrics

• inference time (ms) (primary) — range: ms
  • Wall-clock time to complete DNN inference on the target GPU, measured empirically and compared against model predictions.
• energy consumption (J) — range: J
  • Total energy used during inference, calculated from measured power consumption over the inference duration. Evaluated against fixed energy constraints.
• partition point — range: integer index
  • The index of the DNN block where computation is split between a local device and an edge server to minimize energy while meeting a latency deadline.

Input / output format

Input: DNN architecture (VGG19 or ResNet152), input image size, target GPU frequency, execution deadline, energy constraint, and communication rate (for cooperative inference).

Output: Predicted inference time and energy consumption from the latency model, compared against actual hardware measurements; optimal DNN block partition index for cooperative scenarios.

Scoring recipe

# Compare predicted vs actual hardware measurements
pred_time = model.predict_flops(flops, freq)
actual_time = measure_hardware_time(dnn, freq)
time_error = abs(pred_time - actual_time)

pred_energy = actual_time * power(freq)
actual_energy = measure_hardware_energy(dnn, freq)
energy_reduction = (pred_energy - actual_energy) / pred_energy

# Cooperative: find partition point minimizing energy under deadline
for p in range(num_blocks):
    local_time, comm_time, offload_time = compute_partition_latency(dnn, p, comm_rate)
    if local_time + comm_time + offload_time <= deadline:
        energy = compute_partition_energy(dnn, p, freq, comm_rate)
        if energy < min_energy:
            best_partition = p

Common pitfalls

• Assuming GPU inference time scales linearly with FLOPs, which ignores memory bandwidth and architectural bottlenecks unique to GPUs.
• Using CPU-DVFS power models for GPUs, as GPU power scales differently with frequency and workload intensity.
• Ignoring the cubic relationship between frequency and power (P ∝ f^3) when estimating energy savings from frequency scaling.

Evidence (verbatim from paper)

In Fig. [8], we compare the actual inference time based on these two different models under three given execution time deadlines (i.e., $D=100$ ms, 150 ms, and 200 ms). The optimal is to adjust the GPU frequency to make the inference tasks completed within the given deadlines and reduce the energy consumption as much as possible.

Citation

@misc{han2025dvfs,
  title={DVFS-Aware DNN Inference on GPUs: Latency Modeling and Performance Analysis},
  author={Han et al. (2025)},
  year={2025},
  note={arXiv:2502.06295}
}

• arXiv: 2502.06295