Mgmark Eval

mgmark-eval

MGSim + MGMark: A Framework for Multi-GPU System Research — Sun et al. (2018) (arXiv:1811.02884, 2018)

What this evaluates

Evaluates the cycle-accurate simulation fidelity and performance of a multi-GPU simulator against real hardware. It probes the simulator's ability to model microarchitectural components (ALU, L1/L2 caches, DRAM) and cross-GPU memory access patterns under unified memory systems.

Datasets

• MGMark — total ?; splits: validation (-1), multi-gpu-case-study (-1)

Metrics

• execution time (primary) — range: percent
  • Absolute or relative difference between MGSim-simulated execution cycles/time and real hardware execution time measured via Radeon Compute Profiler. Validated when simulated cycles match real cycles within a small tolerance.
• simulation speedup — range: ratio
  • Ratio of serial simulation time to multi-threaded simulation time, measured by varying the GOMAXPROCS environment variable on the host CPU.

Input / output format

Input: GCN3 assembly kernels with C++ host programs (micro-benchmarks) or ROCm-compiled MGMark workloads (AES, BS, FIR, GD, KM, MT, SC) with specified workload sizes.

Output: Execution time traces, cycle counts, cache/DRAM latency measurements, and speedup ratios compared against real hardware baselines.

Scoring recipe

def compute_accuracy(sim_time, real_time):
    return abs(sim_time - real_time) / real_time

def compute_speedup(serial_time, parallel_time):
    return serial_time / parallel_time

Common pitfalls

• Failing to lock GPU frequency (DVFS) can cause timing variance, invalidating cycle-accurate validation.
• Assuming L1/L2 cache hit/miss behavior without accounting for 64-byte block sizes and specific cache associativity.
• Not quadrupling workload sizes for 4-GPU experiments, leading to under-stressed memory bandwidth and synchronization benchmarks.

Evidence (verbatim from paper)

In order to validate MGSim against a real hardware, we collect the actual GPU execution time as a golden performance reference. The validation system has 2 Intel Xeon E2560 v4 CPUs and one AMD R9 Nano GPU (details provided in Table I). The system runs the Radeon Open Compute Platform (ROCm) 1.7 GPU software stack on a Linux Ubuntu 16.04.4 operating system. We lock the GPUs to run at the maximum frequency to avoid the impact of the Dynamic Frequency and Voltage Scaling (DVFS) on the system. All the timing results are collected using the Radeon Compute Profiler.

Citation

@misc{sun2018mgsim,
  title={MGSim + MGMark: A Framework for Multi-GPU System Research},
  author={Sun et al. (2018)},
  year={2018},
  note={arXiv:1811.02884}
}

• arXiv: 1811.02884