INFOCOM Experiments
Use this before submission when the evaluation is not yet locked. INFOCOM reviewers span the full
analysis → simulation → testbed spectrum, and the evaluation is where a good idea is won or
lost — especially since there is (traditionally) no rebuttal to repair a weak comparison. The
organizing principle is evidence proportional to the claim, delivered by the method the claim
demands: a bound needs a proof, a design needs a fair experiment, a behavior claim needs
measurement.
Evaluation audit
- Match the method to the claim. An optimality/approximation claim needs a proof under
stated assumptions; a performance claim needs a fair experiment; a real-world behavior
claim needs measurement on real traffic. A simulation cannot substitute for a proof, and a
proof under unrealistic assumptions cannot substitute for an experiment.
- State and justify every assumption. For analytical work, list the assumptions (A1, A2, ...)
and defend each; a theorem true only under an assumption no network satisfies is a scored
weakness a reviewer will name — and you cannot rebut it.
- Name the simulator and pin the setup. State whether it is ns-3, OMNeT++, a custom simulator,
or a testbed; report the topology, traffic model, number of runs, seeds, and confidence intervals.
"A large-scale simulation" with no setup is not evidence.
- Choose honest baselines, including the strongest prior scheme and a simple-but-reasonable
alternative, each tuned with a documented, equal budget. An untuned baseline is the classic
INFOCOM reject.
- Report variance, not point estimates. Confidence intervals over multiple seeded runs;
appropriate statistics; say what the intervals represent.
- For testbeds/measurement, describe the hardware, the trace or workload, the collection
methodology, and the confounds — and bound them.
Claim-to-evidence design table
| Networking claim |
Matching evidence |
Reject pattern avoided |
| "Algorithm is near-optimal" |
Proof of an approximation/competitive ratio under stated assumptions |
"Only shown on examples the authors chose" |
| "Scheme improves throughput/delay" |
Seeded simulation or testbed vs. a tuned baseline, with CIs |
"Untuned baseline; single run; no variance" |
| "Scales to large networks" |
Runtime/quality across realistic topology sizes |
"Only a 10-node topology tested" |
| "Holds in real deployments" |
Testbed or trace-driven measurement with real traffic |
"Synthetic traffic claimed to generalize" |
| "The learner adds the value" |
Ablation vs. a non-ML heuristic on the same inputs |
"ML component's marginal value never isolated" |
Simulation and reproducibility floor
[Simulator] name it (ns-3/OMNeT++/custom); pin the version; describe the model, not just the tool
[Topology] real or realistic (e.g., from a topology dataset); state size and generation method
[Traffic] the workload/trace and its source; synthetic models justified against real behavior
[Runs/seeds] multiple seeded runs; report the number and log the seeds; give confidence intervals
[Baselines] the strongest competitor + a simple one, each tuned with a documented equal budget
[Compute] the runtime/scale actually reached, not vague feasibility language
Analytical-work floor
- List assumptions explicitly and justify each against real network conditions.
- Give proof sketches in the body and full proofs in a tight in-budget appendix (it counts toward
the nine pages) or state where the full proof lives.
- Validate the theory in simulation: show the bound is tight/loose where the analysis predicts.
Vignette: evaluating a caching/offloading policy
Suppose the paper claims an online caching policy with a competitive ratio that also beats prior
heuristics. The matching plan: prove the ratio under stated assumptions; then simulate on a real
request trace with multiple seeds, comparing against the strongest prior policy tuned with an equal
budget and a simple LRU baseline; report hit-rate and delay with confidence intervals; show where
the empirical gap matches the analytical bound; and state the regime (skewed vs. uniform demand)
where the guarantee is loose — every number traceable to a logged run.
Statistical reporting floor
- Confidence intervals over seeded runs for every simulated/measured comparison; say what they
represent.
- The number of runs and the source of randomness for any stochastic component.
- The topology sizes and traffic models actually used, not vague "large-scale" language.
Output format
[Evaluation readiness] strong / adequate / weak
[Claim -> method map] <claim: proof | simulation | testbed/measurement>
[Assumptions] <listed and justified? yes/no>
[Simulator/setup] <named? seeds logged? CIs reported? yes/no>
[Baseline fairness] <baseline -> tuned? equal budget? documented?>
[Decision-critical next run] <one experiment or proof to add before the deadline>
Source: brycewang-stanford/Awesome-Journal-Skills → INFOCOM-Skills/skills/infocom-experiments/SKILL.md
1---2name: infocom-experiments3description: Use when designing or auditing IEEE INFOCOM evaluations, covering analytical results with stated and justified assumptions, simulation with a named simulator and logged seeds, testbed and measurement studies with real traffic, honest and tuned baselines, and matching evidence to the shape of each networking claim across the analysis-simulation-testbed spectrum.4---567# INFOCOM Experiments89Use this before submission when the evaluation is not yet locked. INFOCOM reviewers span the full10**analysis → simulation → testbed** spectrum, and the evaluation is where a good idea is won or11lost — especially since there is (traditionally) **no rebuttal** to repair a weak comparison. The12organizing principle is **evidence proportional to the claim**, delivered by the method the claim13demands: a bound needs a proof, a design needs a fair experiment, a behavior claim needs14measurement.1516## Evaluation audit1718- **Match the method to the claim.** An *optimality/approximation* claim needs a **proof** under19 stated assumptions; a *performance* claim needs a **fair experiment**; a *real-world behavior*20 claim needs **measurement** on real traffic. A simulation cannot substitute for a proof, and a21 proof under unrealistic assumptions cannot substitute for an experiment.22- **State and justify every assumption.** For analytical work, list the assumptions (A1, A2, ...)23 and defend each; a theorem true only under an assumption no network satisfies is a scored24 weakness a reviewer will name — and you cannot rebut it.25- **Name the simulator and pin the setup.** State whether it is ns-3, OMNeT++, a custom simulator,26 or a testbed; report the topology, traffic model, number of runs, seeds, and confidence intervals.27 "A large-scale simulation" with no setup is not evidence.28- **Choose honest baselines,** including the strongest prior scheme and a simple-but-reasonable29 alternative, each tuned with a documented, equal budget. An untuned baseline is the classic30 INFOCOM reject.31- **Report variance, not point estimates.** Confidence intervals over multiple seeded runs;32 appropriate statistics; say what the intervals represent.33- **For testbeds/measurement,** describe the hardware, the trace or workload, the collection34 methodology, and the confounds — and bound them.3536## Claim-to-evidence design table3738| Networking claim | Matching evidence | Reject pattern avoided |39|---|---|---|40| "Algorithm is near-optimal" | Proof of an approximation/competitive ratio under stated assumptions | "Only shown on examples the authors chose" |41| "Scheme improves throughput/delay" | Seeded simulation or testbed vs. a tuned baseline, with CIs | "Untuned baseline; single run; no variance" |42| "Scales to large networks" | Runtime/quality across realistic topology sizes | "Only a 10-node topology tested" |43| "Holds in real deployments" | Testbed or trace-driven measurement with real traffic | "Synthetic traffic claimed to generalize" |44| "The learner adds the value" | Ablation vs. a non-ML heuristic on the same inputs | "ML component's marginal value never isolated" |4546## Simulation and reproducibility floor4748```text49[Simulator] name it (ns-3/OMNeT++/custom); pin the version; describe the model, not just the tool50[Topology] real or realistic (e.g., from a topology dataset); state size and generation method51[Traffic] the workload/trace and its source; synthetic models justified against real behavior52[Runs/seeds] multiple seeded runs; report the number and log the seeds; give confidence intervals53[Baselines] the strongest competitor + a simple one, each tuned with a documented equal budget54[Compute] the runtime/scale actually reached, not vague feasibility language55```5657## Analytical-work floor5859- List assumptions explicitly and justify each against real network conditions.60- Give proof sketches in the body and full proofs in a tight in-budget appendix (it counts toward61 the nine pages) or state where the full proof lives.62- Validate the theory in simulation: show the bound is tight/loose where the analysis predicts.6364## Vignette: evaluating a caching/offloading policy6566Suppose the paper claims an online caching policy with a competitive ratio that also beats prior67heuristics. The matching plan: prove the ratio under stated assumptions; then simulate on a real68request trace with multiple seeds, comparing against the strongest prior policy tuned with an equal69budget and a simple LRU baseline; report hit-rate and delay with confidence intervals; show where70the empirical gap matches the analytical bound; and state the regime (skewed vs. uniform demand)71where the guarantee is loose — every number traceable to a logged run.7273## Statistical reporting floor7475- Confidence intervals over seeded runs for every simulated/measured comparison; say what they76 represent.77- The number of runs and the source of randomness for any stochastic component.78- The topology sizes and traffic models actually used, not vague "large-scale" language.7980## Output format8182```text83[Evaluation readiness] strong / adequate / weak84[Claim -> method map] <claim: proof | simulation | testbed/measurement>85[Assumptions] <listed and justified? yes/no>86[Simulator/setup] <named? seeds logged? CIs reported? yes/no>87[Baseline fairness] <baseline -> tuned? equal budget? documented?>88[Decision-critical next run] <one experiment or proof to add before the deadline>89```9091---9293**Source:** [`brycewang-stanford/Awesome-Journal-Skills`](https://github.com/brycewang-stanford/Awesome-Journal-Skills) → `INFOCOM-Skills/skills/infocom-experiments/SKILL.md`
thedixitjain