qhjqhj00 runtime-efficiency-benchmark
Efficient Source Finding for Radio Interferometric Images — Lucas et al. (2019) (arXiv:1901.04956, 2019)
What this evaluates
Evaluates the computational running time, speedup gains, and parallel efficiency of median, standard deviation, and full source-finding algorithms on simulated radio interferometric images.
Datasets
- Simulated Radio Interferometric Images — total ?; splits: (unstated)
Metrics
Running time (seconds)(primary) — range: other- Wall-clock execution time measured in seconds for each algorithmic step or the full pipeline across varying image sizes.
Input / output format
Input: 2D simulated astronomical images of sizes $2^{10}\times 2^{10}$ to $2^{16}\times 2^{16}$, containing 3 Gaussian-shaped sources.
Output: Execution duration in seconds, speedup ratio relative to a baseline implementation, and parallel efficiency percentage.
Scoring recipe
def compute_metrics(algorithm, image_size, baseline_time):
start = time.perf_counter()
result = algorithm.run(image_size)
elapsed = time.perf_counter() - start
speedup = baseline_time / elapsed
parallel_efficiency = (speedup / num_cores) * 100
return elapsed, speedup, parallel_efficiency
Common pitfalls
- Timings vary due to concurrent OS tasks and must be treated as rough estimates rather than absolute constants.
- Small image sizes fit in CPU cache, causing non-linear timing variations unrelated to algorithmic complexity.
- Hyper-threading is explicitly disabled to avoid shared hardware resource contention skewing parallel efficiency.
Evidence (verbatim from paper)
The benchmarking tests discussed here are based on the measured running times, which provide a rough idea of the relative computational complexity of the tested functions. It is important to note that these timings may vary between distinct benchmarking instances, even on the same hardware, as they may be affected by the operating system tasks running concurrently and as well as its status.
Citation
@misc{lucas2019efficient,
title={Efficient Source Finding for Radio Interferometric Images},
author={Lucas et al. (2019)},
year={2019},
note={arXiv:1901.04956}
}
- arXiv: 1901.04956