Load Time

load-time

Survey-scale discovery-based research processes: Evaluating a bespoke visualisation environment for astronomical survey data — Fluke et al. (2023) (arXiv:2307.04357, 2023)

What this evaluates

Evaluates the data loading speed and rendering interactivity of the encube visual analytics framework on a tiled display system under varying data volumes and GPU memory constraints.

Datasets

• Astronomical spectral cube surveys (WHISP, THINGS, LVHIS) — total ?; splits: (unstated)

Metrics

• Load time ($T_{\mathrm{Load}}$) (primary) — range: seconds
  • Total parallel time in seconds to load all spectral cubes from disk to display, measured from UI 'Load' button click until all cubes are visible. Times are rounded up to the nearest whole second.
• Minimum frame rate ($F_{\mathrm{rate}}$) — range: frames/second
  • Steady-state minimum frames per second recorded during a full autospin rotation of all cubes. Measured via S2PLOT debug mode over a 5-second moving window, capturing the worst-case viewing angle.

Input / output format

Input: FITS-format spectral cubes stored on a master node and accessed via NFS, with a JSON configuration file specifying panel layout (rows × columns) and workstation roles.

Output: Numerical performance metrics: total load time (integer seconds) and per-column minimum frame rate (float frames/s).

Scoring recipe

def compute_metrics(config):
    start = time.time()
    trigger_ui_load(config)
    while not all_cubes_displayed(): pass
    load_time = math.ceil(time.time() - start)
    
    trigger_autospin()
    activate_debug_mode()
    min_fps = record_lowest_fps_5s_window()
    return load_time, min_fps

Common pitfalls

• Load times are manually timed and rounded up to the nearest second, introducing human reaction-time bias.
• Frame rates are captured during worst-case viewing angles (full rotation) and may overestimate typical interactive performance.
• GPU memory limits (8 GB) force random sampling with replacement for configurations exceeding survey sizes, altering data distribution.

Evidence (verbatim from paper)

We report: (1) the total (i.e. parallel) load time, $T_{\mathrm{Load}}$, for a configuration displaying $N_{\mathrm{cube}}$ spectral cubes; and (2) the steady-state minimum frame rate, $F_{\mathrm{rate}}$, in frames/second. We consider both the frame rate per column, looking for variations in performance, along with the overall mean, standard deviation, and median of $F_{\mathrm{rate}}$.

Citation

@misc{fluke2023encube,
  title={Survey-scale discovery-based research processes: Evaluating a bespoke visualisation environment for astronomical survey data},
  author={Fluke et al. (2023)},
  year={2023},
  note={arXiv:2307.04357}
}

• arXiv: 2307.04357