Browser Inference Eval

browser_inference_eval

Anatomizing Deep Learning Inference in Web Browsers — Wang et al. (2024) (arXiv:2402.05981, 2024)

What this evaluates

Evaluates the performance overhead and latency characteristics of running deep learning inference directly in web browsers compared to native environments. It probes the impact of WebAssembly runtime inefficiencies, SIMD limitations, and WebGL GPU abstraction on prediction, warmup, and setup phases across various models and hardware configurations.

Datasets

• Inference Benchmark (ResNet50, VGG16, MobileNetV2) — total ?; splits: (unstated)

Metrics

• prediction_latency (primary) — range: ms
  • Mean execution time in milliseconds for the model inference step, measured after framework and model loading. Calculated by averaging prediction times across multiple runs per device/backend configuration.

Input / output format

Input: Pre-trained deep learning models (e.g., ResNet50, VGG16, MobileNetV2) executed via TF.js or ORT.js backends (Wasm/CPU or WebGL/GPU) within a web browser, compared against native TF or ORT implementations.

Output: Latency values in milliseconds for prediction, warmup, and setup phases, reported as averages and quartiles/variance across different devices and backend configurations.

Scoring recipe

def measure_latency(model, backend, device):
    setup_start = time()
    load_framework_and_model(model, backend)
    setup_latency = time() - setup_start
    warmup_start = time()
    run_inference(model, backend, steps=1)
    warmup_latency = time() - warmup_start
    pred_latencies = []
    for _ in range(num_runs):
        pred_start = time()
        run_inference(model, backend)
        pred_latencies.append(time() - pred_start)
    prediction_latency = mean(pred_latencies)
    return {'setup': setup_latency, 'warmup': warmup_latency, 'prediction': prediction_latency}

Common pitfalls

• Confusing warmup latency (shader compilation/memory allocation) with prediction latency, which can skew performance comparisons.
• Ignoring network transfer time when measuring setup latency for in-browser inference, as model files must be downloaded to the browser.
• Overlooking GPU resource contention in WebGL, which causes high latency variance and degrades smoothness despite lower average prediction times.

Evidence (verbatim from paper)

As for prediction latency, the in-browser prediction latency of TF.js is 3.7-18.4× higher than the latency of TF (Table[5] vs. [4]); the gap is 2.1-36.4× between ORT.js and ORT (Table[6] vs. [4]). The average gap of both frameworks is 16.9×.

Citation

@misc{wang2024anatomizing,
  title={Anatomizing Deep Learning Inference in Web Browsers},
  author={Wang et al. (2024)},
  year={2024},
  note={arXiv:2402.05981}
}

• arXiv: 2402.05981