Cosmoflow Hpc Scaling

cosmoflow-hpc-scaling

CosmoFlow: Using Deep Learning to Learn the Universe at Scale — Mathuriya et al. (2018) (arXiv:1808.04728, 2018)

What this evaluates

Evaluates the compute efficiency and horizontal scalability of a 3D convolutional neural network framework on supercomputers, measuring sustained floating-point throughput and parallel scaling efficiency across thousands of nodes.

Datasets

• Cosmological dark matter simulations — total ?; splits: train (-1), validation (-1)

Metrics

• Pflop/s (primary) — range: other
  • Total floating-point operations per epoch divided by the wall-clock time per epoch. The paper reports single-precision performance.
• parallel efficiency — range: percent
  • Speedup on N nodes divided by N, expressed as a percentage relative to single-node performance. Captures end-to-end system capability including I/O and communication.
• throughput — range: other
  • Wall-clock time required to complete one full training epoch, including training loop and loss averaging across MPI ranks.

Input / output format

Input: 3D cosmological simulation data samples (dark matter distributions) processed in mini-batches of size one per node.

Output: Cosmological parameter predictions (Ωₘ, σ₈, nₛ) and scalar loss values for convergence tracking.

Scoring recipe

def compute_sustained_flops(total_flops_per_epoch, walltime_per_epoch):
    return total_flops_per_epoch / walltime_per_epoch

def compute_parallel_efficiency(single_node_time, n_node_time, n_nodes):
    speedup = single_node_time / n_node_time
    return (speedup / n_nodes) * 100

# Note: walltime includes validation and loop overheads.
# Mini-batch size is fixed at 1 per node; global batch scales with nodes.

Common pitfalls

• I/O bottlenecks from shared filesystems (e.g., Lustre) can artificially deflate scaling efficiency if burst buffers are not utilized.
• Walltime measurements include validation and loop overheads, which may differ from pure compute step time and affect reported speedups.
• Fixed mini-batch size of 1 per node means global batch size scales linearly with node count, which can alter convergence dynamics compared to fixed global batch setups.

Evidence (verbatim from paper)

We achieve an average sustained performance of slightly over 3.5 Pflop/s single precision for 8192 nodes with a parallel efficiency of 77% relative to a single node (6324X speedup).

Citation

@misc{mathuriya2018cosmoflow,
  title={CosmoFlow: Using Deep Learning to Learn the Universe at Scale},
  author={Mathuriya et al. (2018)},
  year={2018},
  note={arXiv:1808.04728}
}

• arXiv: 1808.04728