Parma Performance Eval

parma-performance-eval

Parma: Confidential Containers via Attested Execution Policies — Johnson et al. (2023) (arXiv:2302.03976, 2023)

What this evaluates

Measures the computational, network, and database performance overhead of running containerized workloads inside an AMD SEV-SNP enclave with Parma's attested execution policies compared to a baseline outside the enclave.

Datasets

• nginx (wrk2) — total ?; splits: (unstated)
• redis (redis-benchmark) — total ?; splits: (unstated)
• SPEC2017 intspeed — total ?; splits: (unstated)
• NVIDIA Triton Inference Server (perf-analyzer) — total ?; splits: (unstated)

Metrics

• performance_overhead (primary) — range: percent
  • Calculated as the percentage reduction in throughput or increase in latency relative to the baseline configuration. For rate-based metrics: (Baseline_Rate - TEE_Rate) / Baseline_Rate * 100%. For latency/time: (TEE_Value - Baseline_Value) / Baseline_Value * 100%. Reported as mean or median overhead across benchmark runs.

Input / output format

Input: Containerized workloads (web server, in-memory database, CPU benchmarks, ML inference server) deployed on a host machine configured with three setups: baseline (no TEE), SEV-SNP enclave, and SEV-SNP + Parma execution policy.

Output: Raw performance measurements (latency in ms, request/inference rate in req/s, SPEC base ratio) and computed overhead percentages per configuration.

Scoring recipe

def compute_overhead(baseline_val, tee_val, is_rate=True):
    if is_rate:
        return (baseline_val - tee_val) / baseline_val * 100
    else:
        return (tee_val - baseline_val) / baseline_val * 100
# Aggregate median/mean across trials per benchmark.

Common pitfalls

• Overhead varies significantly by workload type (e.g., memory-intensive vs IO-intensive) due to TLB pressure and page table walk metadata checks in SEV-SNP.
• Statistical significance is often low; median latency curves frequently fall within the first quartiles of each other, requiring careful interpretation of aggregate results.
• Baseline configurations must be strictly matched (bare metal vs container vs enclave) to accurately isolate the TEE and policy overhead from containerization overhead.

Evidence (verbatim from paper)

Looking at the geometric mean over all actions, we see a performance overhead of 18% added by operating within the AMD SEV-SNP enclave, and a further 1% when using Parma.

Citation

@misc{johnson2023parma,
  title={Parma: Confidential Containers via Attested Execution Policies},
  author={Johnson et al. (2023)},
  year={2023},
  note={arXiv:2302.03976}
}

• arXiv: 2302.03976