ISSTA Supplementary
Use this when deciding where each piece of content belongs. ISSTA's 18-page limit counts appendices
against the total, so there is no unlimited appendix to absorb overflow: the body must stand on its
own, and the artifact — not a separate PDF supplement — is where extra material goes. Reopen the
current call to confirm whether any in-PDF appendix or separate upload is permitted this cycle.
The split rule
- Everything decision-critical lives inside the 18 pages: the threat model, the technique, the
evaluation design, the headline results, and the threats to validity. If a reviewer needs it to
judge the paper, it cannot live only in the artifact.
- The artifact carries what supports but does not replace the argument: the full tool, complete
per-subject tables, extra configurations, raw logs, and regeneration scripts.
- Do not hide core method steps or primary results in an appendix or the artifact to make the body
fit. Reviewers grade what is in the reviewed pages; buried evidence is invisible evidence.
- Because appendix pages count against 18, a long appendix competes directly with the body. Prefer
moving detail to the artifact over spending page budget on it.
What reviewers open, and what they do not
| Content |
Where it belongs |
Reviewer attention |
| Technique definition, threat model |
Body |
Read closely — graded for soundness |
| Headline results + statistics |
Body |
Read closely — graded for evaluation |
| Threats to validity |
Body |
Expected; its absence is noticed |
| Full per-subject result tables |
Artifact |
Skimmed if a body claim is contested |
| Raw fuzzing/analysis logs |
Artifact |
Rarely opened; ship for completeness |
| The runnable tool |
Artifact (evaluated separately) |
Run by artifact evaluators, not always by PC |
Assume a PC reviewer reads the 18 pages and dips into the artifact only to resolve a doubt. Design
the body so no such dip is required to accept the paper.
Anonymity of supplementary material
- Keep the review-time artifact double-anonymous: no repository owner, commit authors, container
labels, institution paths, or acknowledgements.
- Verify archives open on a clean machine with no credentials, caches, or OS metadata baked in.
- An anonymized public mirror or an anonymous upload is fine; a link to your named GitHub is a
double-anonymous violation.
Vignette: splitting a static-analysis paper
A submission presenting a new taint-analysis with a soundness argument and a large evaluation: the
body keeps the analysis rules, the soundness sketch, the evaluation protocol, and the two
decision-critical result tables (precision/recall against ground truth, and a comparison to one
established baseline); the artifact holds the full implementation, the complete per-benchmark
tables, alternative sensitivity configurations, and the scripts that regenerate every table.
Nothing needed to accept the paper lives only in the artifact, because artifact inspection by the
PC is discretionary.
Body (<= 18 pp) Artifact
- analysis rules - full analyzer implementation
- soundness sketch - complete per-benchmark tables
- evaluation protocol - extra sensitivity configs
- 2 headline tables - table-regeneration scripts + raw logs
- threats to validity - pinned subjects (SHAs)
Output format
[Split status] Ready / Needs fixes / Not ready
[Body carries] <decision-critical items present in the 18 pages>
[Artifact carries] <supporting material>
[Page-budget risk] <appendix competing with the body?>
[Anonymity checks] passed / issues
[Main-paper dependency] <what breaks if the artifact is never opened>
1---2name: issta-supplementary3description: Use when deciding what lives in the ISSTA 18-page body versus the artifact and any appendix, covering the no-unlimited-appendix reality, what reviewers will and will not open, splitting a testing/analysis paper between body and package, anonymity of supplementary material, and keeping decision-critical evidence inside the reviewed pages.4metadata:5 source: https://skillmd.com/skills/brycewang-stanford/issta-supplementary6---78# ISSTA Supplementary910Use this when deciding where each piece of content belongs. ISSTA's 18-page limit counts appendices11against the total, so there is no unlimited appendix to absorb overflow: the body must stand on its12own, and the artifact — not a separate PDF supplement — is where extra material goes. Reopen the13current call to confirm whether any in-PDF appendix or separate upload is permitted this cycle.1415## The split rule1617- Everything decision-critical lives inside the 18 pages: the threat model, the technique, the18 evaluation design, the headline results, and the threats to validity. If a reviewer needs it to19 judge the paper, it cannot live only in the artifact.20- The artifact carries what supports but does not replace the argument: the full tool, complete21 per-subject tables, extra configurations, raw logs, and regeneration scripts.22- Do not hide core method steps or primary results in an appendix or the artifact to make the body23 fit. Reviewers grade what is in the reviewed pages; buried evidence is invisible evidence.24- Because appendix pages count against 18, a long appendix competes directly with the body. Prefer25 moving detail to the artifact over spending page budget on it.2627## What reviewers open, and what they do not2829| Content | Where it belongs | Reviewer attention |30|---|---|---|31| Technique definition, threat model | Body | Read closely — graded for soundness |32| Headline results + statistics | Body | Read closely — graded for evaluation |33| Threats to validity | Body | Expected; its absence is noticed |34| Full per-subject result tables | Artifact | Skimmed if a body claim is contested |35| Raw fuzzing/analysis logs | Artifact | Rarely opened; ship for completeness |36| The runnable tool | Artifact (evaluated separately) | Run by artifact evaluators, not always by PC |3738Assume a PC reviewer reads the 18 pages and dips into the artifact only to resolve a doubt. Design39the body so no such dip is *required* to accept the paper.4041## Anonymity of supplementary material4243- Keep the review-time artifact double-anonymous: no repository owner, commit authors, container44 labels, institution paths, or acknowledgements.45- Verify archives open on a clean machine with no credentials, caches, or OS metadata baked in.46- An anonymized public mirror or an anonymous upload is fine; a link to your named GitHub is a47 double-anonymous violation.4849## Vignette: splitting a static-analysis paper5051A submission presenting a new taint-analysis with a soundness argument and a large evaluation: the52body keeps the analysis rules, the soundness sketch, the evaluation protocol, and the two53decision-critical result tables (precision/recall against ground truth, and a comparison to one54established baseline); the artifact holds the full implementation, the complete per-benchmark55tables, alternative sensitivity configurations, and the scripts that regenerate every table.56Nothing needed to accept the paper lives only in the artifact, because artifact inspection by the57PC is discretionary.5859```text60Body (<= 18 pp) Artifact61- analysis rules - full analyzer implementation62- soundness sketch - complete per-benchmark tables63- evaluation protocol - extra sensitivity configs64- 2 headline tables - table-regeneration scripts + raw logs65- threats to validity - pinned subjects (SHAs)66```6768## Output format6970```text71[Split status] Ready / Needs fixes / Not ready72[Body carries] <decision-critical items present in the 18 pages>73[Artifact carries] <supporting material>74[Page-budget risk] <appendix competing with the body?>75[Anonymity checks] passed / issues76[Main-paper dependency] <what breaks if the artifact is never opened>77```