Iterations Per Minute

iterations-per-minute

The State of the Art when using GPUs in Devising Image Generation Methods Using Deep Learning — Kawahata (2021) (arXiv:2109.05783, 2021)

What this evaluates

Evaluates the computational throughput and hardware scalability of deep learning image generation models by measuring how many training iterations can be completed per minute on CPU versus GPU across varying image resolutions.

Datasets

• Unspecified image set (64/128/256 px) — total ?; splits: (unstated)

Metrics

• Iterations per minute (primary) — range: other
  • Total number of completed training iterations divided by the elapsed time in minutes. Calculated over fixed 1-minute intervals up to 5 minutes.

Input / output format

Input: Image data of fixed pixel dimensions (64, 128, or 256 pixels) processed through VGG or NIN architectures.

Output: Cumulative iteration count recorded at 1-minute intervals up to 5 minutes.

Scoring recipe

def calculate_throughput(cumulative_iterations, elapsed_minutes):
    # cumulative_iterations: list of total iterations at each 1-min mark
    # elapsed_minutes: total duration (e.g., 5)
    return cumulative_iterations[-1] / elapsed_minutes

Common pitfalls

• The metric measures raw computational throughput, not image generation quality or model accuracy.
• Performance drops significantly at 256 pixels and core dumps occur at 512+ pixels due to vectorization bottlenecks, not model complexity.
• cuDNN acceleration was intentionally excluded from these specific benchmark runs to isolate raw GPU vs CPU performance.

Evidence (verbatim from paper)

In this section, we compare (1) the number of Iterations per minute between the GPU and the CPU when the VGG model is used and when the NIN model is used, and (2) the number of Iterations per minute by the number of pixels when the VGG model is used, using an image with 128 pixels.

Citation

@misc{kawahata2021gpuimagegeneration,
  title={The State of the Art when using GPUs in Devising Image Generation Methods Using Deep Learning},
  author={Kawahata (2021)},
  year={2021},
  note={arXiv:2109.05783}
}

• arXiv: 2109.05783