ISSTA Supplementary
Use this when deciding where each piece of content belongs. ISSTA's 18-page limit counts appendices
against the total, so there is no unlimited appendix to absorb overflow: the body must stand on its
own, and the artifact — not a separate PDF supplement — is where extra material goes. Reopen the
current call to confirm whether any in-PDF appendix or separate upload is permitted this cycle.
The split rule
- Everything decision-critical lives inside the 18 pages: the threat model, the technique, the
evaluation design, the headline results, and the threats to validity. If a reviewer needs it to
judge the paper, it cannot live only in the artifact.
- The artifact carries what supports but does not replace the argument: the full tool, complete
per-subject tables, extra configurations, raw logs, and regeneration scripts.
- Do not hide core method steps or primary results in an appendix or the artifact to make the body
fit. Reviewers grade what is in the reviewed pages; buried evidence is invisible evidence.
- Because appendix pages count against 18, a long appendix competes directly with the body. Prefer
moving detail to the artifact over spending page budget on it.
What reviewers open, and what they do not
| Content |
Where it belongs |
Reviewer attention |
| Technique definition, threat model |
Body |
Read closely — graded for soundness |
| Headline results + statistics |
Body |
Read closely — graded for evaluation |
| Threats to validity |
Body |
Expected; its absence is noticed |
| Full per-subject result tables |
Artifact |
Skimmed if a body claim is contested |
| Raw fuzzing/analysis logs |
Artifact |
Rarely opened; ship for completeness |
| The runnable tool |
Artifact (evaluated separately) |
Run by artifact evaluators, not always by PC |
Assume a PC reviewer reads the 18 pages and dips into the artifact only to resolve a doubt. Design
the body so no such dip is required to accept the paper.
Anonymity of supplementary material
- Keep the review-time artifact double-anonymous: no repository owner, commit authors, container
labels, institution paths, or acknowledgements.
- Verify archives open on a clean machine with no credentials, caches, or OS metadata baked in.
- An anonymized public mirror or an anonymous upload is fine; a link to your named GitHub is a
double-anonymous violation.
Vignette: splitting a static-analysis paper
A submission presenting a new taint-analysis with a soundness argument and a large evaluation: the
body keeps the analysis rules, the soundness sketch, the evaluation protocol, and the two
decision-critical result tables (precision/recall against ground truth, and a comparison to one
established baseline); the artifact holds the full implementation, the complete per-benchmark
tables, alternative sensitivity configurations, and the scripts that regenerate every table.
Nothing needed to accept the paper lives only in the artifact, because artifact inspection by the
PC is discretionary.
Body (<= 18 pp) Artifact
- analysis rules - full analyzer implementation
- soundness sketch - complete per-benchmark tables
- evaluation protocol - extra sensitivity configs
- 2 headline tables - table-regeneration scripts + raw logs
- threats to validity - pinned subjects (SHAs)
Output format
[Split status] Ready / Needs fixes / Not ready
[Body carries] <decision-critical items present in the 18 pages>
[Artifact carries] <supporting material>
[Page-budget risk] <appendix competing with the body?>
[Anonymity checks] passed / issues
[Main-paper dependency] <what breaks if the artifact is never opened>
1---2name: issta-supplementary3description: Use when deciding what lives in the ISSTA 18-page body versus the artifact and any appendix, covering the no-unlimited-appendix reality, what reviewers will and will not open, splitting a testing/analysis paper between body and package, anonymity of supplementary material, and keeping decision-critical evidence inside the reviewed pages.4---56# ISSTA Supplementary78Use this when deciding where each piece of content belongs. ISSTA's 18-page limit counts appendices9against the total, so there is no unlimited appendix to absorb overflow: the body must stand on its10own, and the artifact — not a separate PDF supplement — is where extra material goes. Reopen the11current call to confirm whether any in-PDF appendix or separate upload is permitted this cycle.1213## The split rule1415- Everything decision-critical lives inside the 18 pages: the threat model, the technique, the16 evaluation design, the headline results, and the threats to validity. If a reviewer needs it to17 judge the paper, it cannot live only in the artifact.18- The artifact carries what supports but does not replace the argument: the full tool, complete19 per-subject tables, extra configurations, raw logs, and regeneration scripts.20- Do not hide core method steps or primary results in an appendix or the artifact to make the body21 fit. Reviewers grade what is in the reviewed pages; buried evidence is invisible evidence.22- Because appendix pages count against 18, a long appendix competes directly with the body. Prefer23 moving detail to the artifact over spending page budget on it.2425## What reviewers open, and what they do not2627| Content | Where it belongs | Reviewer attention |28|---|---|---|29| Technique definition, threat model | Body | Read closely — graded for soundness |30| Headline results + statistics | Body | Read closely — graded for evaluation |31| Threats to validity | Body | Expected; its absence is noticed |32| Full per-subject result tables | Artifact | Skimmed if a body claim is contested |33| Raw fuzzing/analysis logs | Artifact | Rarely opened; ship for completeness |34| The runnable tool | Artifact (evaluated separately) | Run by artifact evaluators, not always by PC |3536Assume a PC reviewer reads the 18 pages and dips into the artifact only to resolve a doubt. Design37the body so no such dip is *required* to accept the paper.3839## Anonymity of supplementary material4041- Keep the review-time artifact double-anonymous: no repository owner, commit authors, container42 labels, institution paths, or acknowledgements.43- Verify archives open on a clean machine with no credentials, caches, or OS metadata baked in.44- An anonymized public mirror or an anonymous upload is fine; a link to your named GitHub is a45 double-anonymous violation.4647## Vignette: splitting a static-analysis paper4849A submission presenting a new taint-analysis with a soundness argument and a large evaluation: the50body keeps the analysis rules, the soundness sketch, the evaluation protocol, and the two51decision-critical result tables (precision/recall against ground truth, and a comparison to one52established baseline); the artifact holds the full implementation, the complete per-benchmark53tables, alternative sensitivity configurations, and the scripts that regenerate every table.54Nothing needed to accept the paper lives only in the artifact, because artifact inspection by the55PC is discretionary.5657```text58Body (<= 18 pp) Artifact59- analysis rules - full analyzer implementation60- soundness sketch - complete per-benchmark tables61- evaluation protocol - extra sensitivity configs62- 2 headline tables - table-regeneration scripts + raw logs63- threats to validity - pinned subjects (SHAs)64```6566## Output format6768```text69[Split status] Ready / Needs fixes / Not ready70[Body carries] <decision-critical items present in the 18 pages>71[Artifact carries] <supporting material>72[Page-budget risk] <appendix competing with the body?>73[Anonymity checks] passed / issues74[Main-paper dependency] <what breaks if the artifact is never opened>75```
brycewang-stanford