UIST Artifact Evaluation
UIST has no artifact-evaluation committee or badge track (none was found for the
2026 cycle — 待核实 each year); the CFP-level instrument of proof is the video
figure. That absence raises rather than lowers the packaging bar: your artifacts are
judged twice, informally — at review time as evidence the system exists as
claimed, and after publication as infrastructure other builders adopt. Nobody
will certify either; both simply succeed or fail.
What counts as the artifact, by paper type
| Paper type |
Review-time artifact |
Reuse-time artifact |
| Interaction technique |
Reference implementation + demo scene |
Portable library with the technique isolated |
| Toolkit / authoring system |
Runnable toolkit + the example apps from the paper |
Documented API, tutorials, package registry entry |
| Hardware / fabrication |
Design files, firmware, BOM, assembly photos |
Fab-ready files + sourcing notes + calibration guide |
| Sensing / recognition pipeline |
Trained models + capture data + eval harness |
Dataset with collection protocol + retraining scripts |
| Human-AI / LLM system |
Prompts, orchestration code, pinned model IDs, logged transcripts |
Same, plus cost and drift notes |
Review-time packaging: the five-minute skeptic
A reviewer gives your supplement five minutes, anonymously, on a machine you don't
control. Optimize for that reader:
- One
README at the archive root: what this is, which paper section each
directory backs, and one command (or one video) per claim.
- Prefer a recorded run alongside the code for anything with hardware, drivers,
or GPU dependencies — reviewers cannot rebuild your rig, so show the harness
producing the paper's numbers.
- Pin everything (lockfiles, container image digests, model checkpoints); "latest"
is a broken artifact by review week.
- Anonymize as strictly as the PDF: repository history, notebook authorship cells,
hardcoded home paths, calibration files named after lab members (see
uist-submission for the sweep).
supplement.zip
├── README.md # claim → artifact map; 5-minute quickstart
├── technique/ # core implementation, pinned deps
├── hardware/ # schematics, PCB, STL/STEP, BOM.csv, firmware/
├── eval/ # harness + raw logs behind Tables 1-2
│ └── rerun.sh # regenerates the paper's numbers from logs
├── media/ # per-claim capture clips (beyond the video figure)
└── LICENSES.md # third-party components and their terms
Release-time packaging: engineering for strangers
At camera-ready (see uist-camera-ready), the audience flips from three skeptics
to an open-ended stream of builders:
- De-anonymize deliberately — publish to the real org, restore attribution,
add the paper citation and BibTeX to the README.
- Cut a release tag matching the camera-ready ("as-published") so later
development never orphans the paper's claims.
- Choose licenses by artifact class: code (e.g. MIT/Apache-2.0), hardware
designs (e.g. CERN-OHL), data (e.g. CC-BY) — one archive often needs all three,
and institutional tech-transfer rules for hardware are worth checking early.
- Archive beyond the repo: deposit the tagged release with a DOI service so
the URL in the proceedings outlives your hosting choices.
- State the support posture honestly in the README — "research prototype,
issues welcome, no maintenance promised" is respectable; silence is not.
What the informal evaluators open first
Order the package for actual reading behavior:
- README, thirty seconds. If the claim → artifact map is not visible without
scrolling, the evaluation is over.
- The media directory, two minutes. Clips of the harness producing the
paper's numbers get watched; they are the highest-credibility artifact per
byte, especially for hardware.
- One quickstart command, two minutes. Whatever you name in the README as
"run this" will be run in a fresh environment; test it in a container or a
colleague's clean machine, not your dev box.
- Source spot-checks. Reviewers grep for the mechanism the paper claims is
novel; if the "self-calibrating controller" is a 30-line stub, the paper's
credibility inverts. Never ship scaffolding that contradicts the prose.
Toolkit papers: adoption is the long evaluation
For toolkit and authoring-system contributions, the release is the deferred
evaluation, and small engineering choices compound:
- Publish to the ecosystem's registry (pip/npm/crates/Arduino library manager) —
installability is adoption's first filter.
- Ship the paper's example applications as runnable starters; they are the
tutorials people actually read.
- Keep the API surface documented at the level of the paper's abstractions, so
citations of the toolkit describe your concepts in your vocabulary.
- Track downstream uses; a "built with X" list is both maintenance motivation
and the evidence base for the retrospective the venue's decade-scale memory
eventually invites.
Hardware honesty
Physical artifacts cannot be uploaded, so their evidence standard is
reconstruction: exact part numbers with sources, tolerances that matter, assembly
sequence photos, firmware flashing instructions, and the calibration procedure with
expected readings. A paper whose device only the authors can build has published a
demo, not a contribution — reviewers from fabrication-heavy labs apply exactly that
test (see uist-reproducibility for the replication ledger).
Output format
[Artifact class] technique / toolkit / hardware / pipeline / hybrid
[Review package] five-minute test passes? claim→artifact map complete?
[Anonymity] archive-level sweep clean?
[Release plan] tag · licenses (code/hardware/data) · DOI deposit · support posture
[Gap list] <artifacts named in the paper but absent from the package>
1---2name: uist-artifact-evaluation3description: Use when packaging the artifacts behind a UIST paper — code, toolkits, hardware design files, and datasets — first as anonymous review-time evidence that the system is real, then as a public release engineered for reuse, in a venue with no formal badge committee doing the checking for you.4metadata:5 source: https://skillmd.com/skills/brycewang-stanford/uist-artifact-evaluation6---78# UIST Artifact Evaluation910UIST has no artifact-evaluation committee or badge track (none was found for the112026 cycle — 待核实 each year); the CFP-level instrument of proof is the video12figure. That absence raises rather than lowers the packaging bar: your artifacts are13judged twice, informally — at review time as *evidence the system exists as14claimed*, and after publication as *infrastructure other builders adopt*. Nobody15will certify either; both simply succeed or fail.1617## What counts as the artifact, by paper type1819| Paper type | Review-time artifact | Reuse-time artifact |20|---|---|---|21| Interaction technique | Reference implementation + demo scene | Portable library with the technique isolated |22| Toolkit / authoring system | Runnable toolkit + the example apps from the paper | Documented API, tutorials, package registry entry |23| Hardware / fabrication | Design files, firmware, BOM, assembly photos | Fab-ready files + sourcing notes + calibration guide |24| Sensing / recognition pipeline | Trained models + capture data + eval harness | Dataset with collection protocol + retraining scripts |25| Human-AI / LLM system | Prompts, orchestration code, pinned model IDs, logged transcripts | Same, plus cost and drift notes |2627## Review-time packaging: the five-minute skeptic2829A reviewer gives your supplement five minutes, anonymously, on a machine you don't30control. Optimize for that reader:3132- One `README` at the archive root: what this is, which paper section each33 directory backs, and one command (or one video) per claim.34- Prefer a **recorded run alongside the code** for anything with hardware, drivers,35 or GPU dependencies — reviewers cannot rebuild your rig, so show the harness36 producing the paper's numbers.37- Pin everything (lockfiles, container image digests, model checkpoints); "latest"38 is a broken artifact by review week.39- Anonymize as strictly as the PDF: repository history, notebook authorship cells,40 hardcoded home paths, calibration files named after lab members (see41 `uist-submission` for the sweep).4243```text44supplement.zip45├── README.md # claim → artifact map; 5-minute quickstart46├── technique/ # core implementation, pinned deps47├── hardware/ # schematics, PCB, STL/STEP, BOM.csv, firmware/48├── eval/ # harness + raw logs behind Tables 1-249│ └── rerun.sh # regenerates the paper's numbers from logs50├── media/ # per-claim capture clips (beyond the video figure)51└── LICENSES.md # third-party components and their terms52```5354## Release-time packaging: engineering for strangers5556At camera-ready (see `uist-camera-ready`), the audience flips from three skeptics57to an open-ended stream of builders:58591. **De-anonymize deliberately** — publish to the real org, restore attribution,60 add the paper citation and BibTeX to the README.612. **Cut a release tag** matching the camera-ready ("as-published") so later62 development never orphans the paper's claims.633. **Choose licenses by artifact class**: code (e.g. MIT/Apache-2.0), hardware64 designs (e.g. CERN-OHL), data (e.g. CC-BY) — one archive often needs all three,65 and institutional tech-transfer rules for hardware are worth checking early.664. **Archive beyond the repo**: deposit the tagged release with a DOI service so67 the URL in the proceedings outlives your hosting choices.685. **State the support posture** honestly in the README — "research prototype,69 issues welcome, no maintenance promised" is respectable; silence is not.7071## What the informal evaluators open first7273Order the package for actual reading behavior:74751. **README, thirty seconds.** If the claim → artifact map is not visible without76 scrolling, the evaluation is over.772. **The media directory, two minutes.** Clips of the harness producing the78 paper's numbers get watched; they are the highest-credibility artifact per79 byte, especially for hardware.803. **One quickstart command, two minutes.** Whatever you name in the README as81 "run this" will be run in a fresh environment; test it in a container or a82 colleague's clean machine, not your dev box.834. **Source spot-checks.** Reviewers grep for the mechanism the paper claims is84 novel; if the "self-calibrating controller" is a 30-line stub, the paper's85 credibility inverts. Never ship scaffolding that contradicts the prose.8687## Toolkit papers: adoption is the long evaluation8889For toolkit and authoring-system contributions, the release *is* the deferred90evaluation, and small engineering choices compound:9192- Publish to the ecosystem's registry (pip/npm/crates/Arduino library manager) —93 installability is adoption's first filter.94- Ship the paper's example applications as runnable starters; they are the95 tutorials people actually read.96- Keep the API surface documented at the level of the paper's abstractions, so97 citations of the toolkit describe your concepts in your vocabulary.98- Track downstream uses; a "built with X" list is both maintenance motivation99 and the evidence base for the retrospective the venue's decade-scale memory100 eventually invites.101102## Hardware honesty103104Physical artifacts cannot be uploaded, so their evidence standard is105reconstruction: exact part numbers with sources, tolerances that matter, assembly106sequence photos, firmware flashing instructions, and the calibration procedure with107expected readings. A paper whose device only the authors can build has published a108demo, not a contribution — reviewers from fabrication-heavy labs apply exactly that109test (see `uist-reproducibility` for the replication ledger).110111## Output format112113```text114[Artifact class] technique / toolkit / hardware / pipeline / hybrid115[Review package] five-minute test passes? claim→artifact map complete?116[Anonymity] archive-level sweep clean?117[Release plan] tag · licenses (code/hardware/data) · DOI deposit · support posture118[Gap list] <artifacts named in the paper but absent from the package>119```