CoNEXT Experiments
Build the evaluation a networking reviewer will actually interrogate. CoNEXT's evidence culture is
systems-and-measurement: claims are backed on the real target platform — a testbed,
deployment, or trace — with honest baselines and reported uncertainty, not simulation standing in
for hardware or a single number with no variance. Because a one-shot major revision is decided on a
list of minimum necessary changes, an evaluation gap you leave now often becomes a mandatory fix
under a tight window later.
Match evidence to claim shape
| Claim shape |
Evidence CoNEXT expects |
| A mechanism is faster/cheaper on real hardware |
A run on the real target (switch, NIC, kernel, testbed) under its real constraints, vs. a tuned baseline, with effect sizes |
| A phenomenon exists in the wild |
A measurement campaign with documented vantage points, capture dates, and a reproducible extraction methodology |
| An architecture scales |
Scalability evidence (real deployment or faithful emulation) across the relevant range, not a point claim |
| An operator intervention helps |
Evidence at operationally relevant scale, with the counterfactual measured or bounded |
| A learned component adds value |
An ablation isolating the learned part from the mechanism, plus a contamination check |
Real testbeds and deployments over simulation
- Evaluate on the platform the paper is about. If the claim is about switch behavior, run it on
the switch (or a faithful hardware testbed); if about a protocol on real paths, use a testbed or
trace-driven replay. Simulation-only systems claims are a classic CoNEXT revision risk.
- Describe the testbed so it can be rebuilt: topology, hardware models, firmware/OS versions,
link rates, buffer depths, and background traffic. A reviewer who cannot picture the setup cannot
trust the numbers.
- State what is emulated vs. real, and why the emulation is faithful for the claim.
Honest, tuned baselines
- Compare against the strongest reasonable baseline, tuned with an equal, documented budget —
not a strawman default. "The baseline is not tuned/fair" is one of the most common CoNEXT
push-backs.
- Report the baseline's configuration alongside yours so the comparison is auditable.
- Where a fair baseline does not exist, say so and justify the comparison you made.
Measurement statistics and uncertainty
- Report multiple runs and variability (confidence intervals, percentiles/CDFs for latency,
not just means). Tail behavior is often the point in networking.
- Use corrected comparisons when testing many conditions; a measurement reviewer notices
uncorrected multiple testing.
- Prefer effect sizes and distributions over a single bar; a CDF that shows the whole
distribution pre-empts questions a mean hides.
Trace and config provenance (pin it now)
- Record capture vantage points and dates, the extraction/anonymization methodology, and the
exact configs used. These cannot be reconstructed after the fact — pin them at collection time.
- For anything derived from operator or third-party data, document the access terms and what can
be released (this interacts with double-anonymity and reproducibility).
- Keep the raw-to-result pipeline scripted so the numbers in the paper regenerate from the trace.
Contamination-aware ML-for-networking ablations
If the paper uses a learner or LLM on networking data:
- Isolate the learned component with an ablation (e.g., replace it with a heuristic) so the
reviewer sees its marginal value over the mechanism.
- Guard against contamination/leakage: ensure test paths/traces/time windows are disjoint from
training, and that a temporal split reflects real deployment.
- Pin model identifiers and dates and cache raw model outputs; a package that needs live API
calls re-samples rather than reproduces (see
conext-reproducibility).
- Apply the model-swap test: if the networking lesson does not survive swapping the model, the
paper may belong at an ML venue (see
conext-topic-selection).
Pre-submission evaluation audit
[Claim coverage] every claim has a matching measurement on the real target? yes/no
[Platform realism] real hardware/testbed/trace, or simulation standing in? note each
[Baselines] strongest reasonable, equally tuned, config reported? yes/no
[Uncertainty] multiple runs, CIs/CDFs, corrected comparisons? yes/no
[Provenance] vantage points, dates, configs, firmware/OS pinned? yes/no
[ML checks] ablation + contamination guard + model-swap survives? n/a or yes/no
Output format
[Evaluation status] solid / gaps
[Claim-evidence matrix] <claim -> measurement + platform + baseline + uncertainty>
[Platform] real target used? emulation justified?
[Provenance] traces/configs/firmware pinned for reproducibility
[Revision risk] <the gap most likely to become a minimum-necessary change>
1---2name: conext-experiments3description: Use when designing or auditing the evaluation of an ACM CoNEXT paper — matching evidence to claim shape with real testbeds and deployments, honest and tuned baselines, measurement statistics and uncertainty, trace and config provenance, and contamination-aware ablations for ML-for-networking work.4metadata:5 source: https://skillmd.com/skills/brycewang-stanford/conext-experiments6---78# CoNEXT Experiments910Build the evaluation a networking reviewer will actually interrogate. CoNEXT's evidence culture is11**systems-and-measurement**: claims are backed on the **real target platform** — a testbed,12deployment, or trace — with honest baselines and reported uncertainty, not simulation standing in13for hardware or a single number with no variance. Because a one-shot major revision is decided on a14list of **minimum necessary changes**, an evaluation gap you leave now often becomes a mandatory fix15under a tight window later.1617## Match evidence to claim shape1819| Claim shape | Evidence CoNEXT expects |20|---|---|21| A mechanism is faster/cheaper on real hardware | A run on the **real target** (switch, NIC, kernel, testbed) under its real constraints, vs. a tuned baseline, with effect sizes |22| A phenomenon exists in the wild | A **measurement campaign** with documented vantage points, capture dates, and a reproducible extraction methodology |23| An architecture scales | **Scalability evidence** (real deployment or faithful emulation) across the relevant range, not a point claim |24| An operator intervention helps | Evidence at **operationally relevant scale**, with the counterfactual measured or bounded |25| A learned component adds value | An **ablation** isolating the learned part from the mechanism, plus a contamination check |2627## Real testbeds and deployments over simulation2829- **Evaluate on the platform the paper is about.** If the claim is about switch behavior, run it on30 the switch (or a faithful hardware testbed); if about a protocol on real paths, use a testbed or31 trace-driven replay. Simulation-only systems claims are a classic CoNEXT revision risk.32- **Describe the testbed so it can be rebuilt:** topology, hardware models, firmware/OS versions,33 link rates, buffer depths, and background traffic. A reviewer who cannot picture the setup cannot34 trust the numbers.35- **State what is emulated vs. real**, and why the emulation is faithful for the claim.3637## Honest, tuned baselines3839- Compare against the **strongest reasonable baseline**, tuned with an equal, documented budget —40 not a strawman default. "The baseline is not tuned/fair" is one of the most common CoNEXT41 push-backs.42- Report the baseline's configuration alongside yours so the comparison is auditable.43- Where a fair baseline does not exist, say so and justify the comparison you made.4445## Measurement statistics and uncertainty4647- Report **multiple runs** and **variability** (confidence intervals, percentiles/CDFs for latency,48 not just means). Tail behavior is often the point in networking.49- Use **corrected comparisons** when testing many conditions; a measurement reviewer notices50 uncorrected multiple testing.51- Prefer **effect sizes and distributions** over a single bar; a CDF that shows the whole52 distribution pre-empts questions a mean hides.5354## Trace and config provenance (pin it now)5556- Record **capture vantage points and dates**, the **extraction/anonymization methodology**, and the57 **exact configs** used. These cannot be reconstructed after the fact — pin them at collection time.58- For anything derived from operator or third-party data, document the **access terms** and what can59 be released (this interacts with double-anonymity and reproducibility).60- Keep the raw-to-result pipeline scripted so the numbers in the paper regenerate from the trace.6162## Contamination-aware ML-for-networking ablations6364If the paper uses a learner or LLM on networking data:6566- **Isolate the learned component** with an ablation (e.g., replace it with a heuristic) so the67 reviewer sees its marginal value over the mechanism.68- **Guard against contamination/leakage:** ensure test paths/traces/time windows are disjoint from69 training, and that a temporal split reflects real deployment.70- **Pin model identifiers and dates** and cache raw model outputs; a package that needs live API71 calls re-samples rather than reproduces (see [`conext-reproducibility`](../conext-reproducibility/SKILL.md)).72- Apply the **model-swap test**: if the networking lesson does not survive swapping the model, the73 paper may belong at an ML venue (see [`conext-topic-selection`](../conext-topic-selection/SKILL.md)).7475## Pre-submission evaluation audit7677```text78[Claim coverage] every claim has a matching measurement on the real target? yes/no79[Platform realism] real hardware/testbed/trace, or simulation standing in? note each80[Baselines] strongest reasonable, equally tuned, config reported? yes/no81[Uncertainty] multiple runs, CIs/CDFs, corrected comparisons? yes/no82[Provenance] vantage points, dates, configs, firmware/OS pinned? yes/no83[ML checks] ablation + contamination guard + model-swap survives? n/a or yes/no84```8586## Output format8788```text89[Evaluation status] solid / gaps90[Claim-evidence matrix] <claim -> measurement + platform + baseline + uncertainty>91[Platform] real target used? emulation justified?92[Provenance] traces/configs/firmware pinned for reproducibility93[Revision risk] <the gap most likely to become a minimum-necessary change>94```