Gpu Memory Co Optimization Eval

gpu-memory-co-optimization-eval

Co-Optimizing Performance and Memory FootprintVia Integrated CPU/GPU Memory Management, anImplementation on Autonomous Driving Platform — Bateni et al. (2020) (arXiv:2003.07945, 2020)

What this evaluates

Evaluates the trade-off between system memory footprint and task latency when co-executing multiple workloads under different integrated CPU/GPU memory management policies on embedded platforms. It measures how strategically assigning Device, Managed, or Host-Pinned memory policies affects peak memory consumption, average GPU execution time, and overall GPU utilization during multitasking.

Datasets

• Rodinia Benchmark Suite (subset) — total ?; splits: test (7)
• DJI Drone Object Detection — total ?; splits: test (1)
• Autoware Perception Module — total ?; splits: test (1)

Metrics

• memory usage — range: other
  • Peak GPU memory consumption measured during concurrent execution of workloads under a given MM policy.
• GPU time (primary) — range: other
  • Average execution time across multiple input data sizes for each benchmark or application.
• GPU utilization — range: percent
  • Percentage of GPU compute capacity actively used during multitasking workloads.
• prediction error — range: percent
  • Geometric mean of the absolute percentage difference between measured and model-predicted GPU times.

Input / output format

Input: Workloads (benchmarks or real applications) with specific input data sizes, scheduled concurrently on embedded SoCs (TX2, AGX, PX2) under three GPU memory management configurations: Default (D), Memory Optimized (MO), and Co-Optimized (CO).

Output: Measured peak memory usage, average GPU execution time, GPU utilization percentage, and model prediction error rates.

Scoring recipe

def evaluate(workloads, config, hardware):
    results = []
    for wl in workloads:
        mem = measure_peak_memory(wl, config)
        time = measure_avg_gpu_time(wl, config)
        util = measure_gpu_utilization(wl, config)
        results.append({'memory': mem, 'time': time, 'util': util})
    errors = []
    for r in results:
        pred = predict_time(r['input_size'], config)
        errors.append(abs(r['time'] - pred) / r['time'])
    error_rate = geometric_mean(errors)
    return results, error_rate

Common pitfalls

• Confusing the three GPU MM policies (Default/Device, Managed, Host-Pinned) and their distinct memory/latency trade-offs.
• Assuming the Memory Optimized (MO) configuration always yields the lowest memory footprint in multitasking, ignoring that the Co-Optimized (CO) policy strategically mixes policies to balance memory and latency.
• Overlooking hardware architecture differences (Parker/TX2 vs. Xavier/AGX), which significantly alter policy selection and performance due to varying memory access and transfer speeds.

Evidence (verbatim from paper)

We report the results of the memory usage and GPU time under three GPU MM setups (D, MO, CO) and two hardware configurations (TX2 and AGX). ... in worst case, the geometric mean of the error is 14.9% for D policy, 16.4% for M policy and 23.1 % for H policy; in average case, the error is 9.6 % for D, 11.8% for M and 6.5% for H policy.

Citation

@misc{bateni2020gpumemory,
  title={Co-Optimizing Performance and Memory FootprintVia Integrated CPU/GPU Memory Management, anImplementation on Autonomous Driving Platform},
  author={Bateni et al. (2020)},
  year={2020},
  note={arXiv:2003.07945}
}

• arXiv: 2003.07945