Secure Inference Latency

secure-inference-latency

Gazelle: A Low Latency Framework for Secure Neural Network Inference — Juvekar et al. (2018) (arXiv:1801.05507, 2018)

What this evaluates

Measures the online and offline computation latency and communication bandwidth for cryptographic primitives and neural network operations under secure two-party computation. It evaluates how efficiently packed homomorphic encryption and garbled circuits handle matrix-vector products, convolutions, and activation functions without revealing inputs or model parameters.

Datasets

• (no dataset; pure metric skill)

Metrics

• t_online (primary) — range: milliseconds
  • Wall-clock time in milliseconds for the online phase of the secure computation protocol, which runs per inference and depends on client input.
• t_setup — range: milliseconds
  • Wall-clock time in milliseconds for the one-time offline setup phase (e.g., key generation, matrix preprocessing) that is amortized across multiple inferences.
• BW_online — range: megabytes
  • Communication bandwidth in megabytes transferred between the client and server during the online phase.
• BW_offline — range: megabytes
  • Communication bandwidth in megabytes transferred during the offline setup phase.

Input / output format

Input: Plaintext matrices/vectors, encrypted ciphertexts, and neural network layer configurations (e.g., filter size, stride, window size, number of SIMD slots).

Output: Execution time (ms) and communication bandwidth (MB) for online and offline phases, along with operation counts (e.g., #in_rot, #out_rot, #mac) and speedup ratios compared to baseline implementations.

Scoring recipe

import time

def evaluate_gazelle(operation_func, num_runs=10):
    start_setup = time.time()
    operation_func.setup()
    t_setup = (time.time() - start_setup) * 1000
    
    start_online = time.time()
    operation_func.run_online()
    t_online = (time.time() - start_online) * 1000
    
    return {
        't_setup_ms': t_setup,
        't_online_ms': t_online,
        'BW_offline_MB': setup_bytes / (1024*1024),
        'BW_online_MB': online_bytes / (1024*1024)
    }

Common pitfalls

• Offline setup time is a one-time cost and should not be averaged or reported as per-inference latency.
• Experiments are conducted in a LAN setting; results will not reflect real-world wide-area network latency or packet loss.
• Ciphertext size is fixed at 32 kB per parameter for the reported bandwidth calculations, so bandwidth scales linearly with the number of packed slots.

Evidence (verbatim from paper)

All benchmarks were generated using c4.xlarge AWS instances which provide a 4-threaded execution environment (on an Intel Xeon E5-2666 v3 2.90GHz CPU) with 7.5GB of system memory. Our experiments were conducted using Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-1041-aws) and our library was compiled using GCC 5.4.0 using the ’-O3’ optimization setting and enabling support for the AES-NI instruction set. Our schemes are evaluated in the LAN setting similar to previous work with both instances in the us-east-1a availability zone. Table [VIII] shows the online computation time and the time required to setup the scheme in milliseconds. | Algorithm | Outputs | t_offline | t_online | BW_offline | BW_online |

Citation

@misc{juvekar2018gazelle,
  title={Gazelle: A Low Latency Framework for Secure Neural Network Inference},
  author={Juvekar et al. (2018)},
  year={2018},
  note={arXiv:1801.05507}
}

• arXiv: 1801.05507