ICASSP Experiments
Use this before submission when the empirical story is not yet locked. ICASSP reviewers are
subfield experts who know the right metric and the right baseline for your task, so the fastest
route to rejection is the wrong ruler or a stale comparison. The four pages force a small
number of decisive experiments, not a large number of weak ones.
Experiment audit
- Map each empirical claim to a specific table, figure, or condition sweep.
- Use the field-standard metric for the task; a novel or convenient metric invites the
"that is not how this task is measured" review.
- Anchor to a current strong baseline and a standard corpus/benchmark, not to a weak or
dated reference that flatters the result.
- Sweep the operating condition that matters (SNR, reverberation, bit rate, noise level); a
single-condition number rarely convinces a signal reviewer.
- Report spread over runs (multiple seeds), and say in the caption whether bars are standard
deviations, standard errors, or confidence intervals.
- Audit for train/test leakage, speaker/scene overlap across splits, and metric computed on the
wrong crop, alignment, or normalization.
Match the metric to the task law
| Task |
Standard metric(s) |
Standard evaluation anchor |
| Speech recognition |
WER / CER |
LibriSpeech, WSJ, or task corpus with fixed split |
| Enhancement / separation |
SI-SDR, PESQ, STOI |
Matched mixture set, reference-aligned scorer |
| Speaker / language ID |
EER, minDCF |
Standard trial lists (e.g., VoxCeleb-style) |
| Sound event / audio tagging |
mAP, F1, error rate |
Fixed labeled set, defined operating point |
| Image / video restoration |
PSNR, SSIM |
Standard test set, defined borders and depth |
| Communications |
BER / BLER vs SNR |
Defined channel model and decoder |
| Estimation / detection |
RMSE, ROC/AUC |
Monte-Carlo trials, bound (Cramér-Rao) if apt |
Reporting the wrong metric family (e.g., classification accuracy for a separation paper) is a
first-round reject pattern; match the ruler to the task before anything else.
What experiments are for at this venue
- ICASSP experiments exist to demonstrate a signal-processing mechanism works under realistic
conditions, not to top a leaderboard by any margin. One clean condition sweep beats five
extra datasets at a single point.
- The strongest design isolates the claimed mechanism with an ablation and shows it holds
across the operating range, with the standard baseline drawn on the same axes.
- Where a theoretical bound exists (estimation, detection, coding), compare against it rather than
only against another method.
Ablation and sweep stub
Fig. 2: metric vs condition (e.g., SI-SDR vs input SNR, 0-20 dB)
- proposed (mean ± sd over 3 seeds)
- strong baseline (same corpus, same scorer)
Table 1: ablation — remove one component at a time, same protocol
- full method | -component A | -component B | baseline
Report: corpus + split, scorer config, seeds, run count, hardware/runtime
Vignette: a dereverberation paper
A submission claims improved dereverberation. The matching plan: evaluate on a standard reverberant
set with PESQ and STOI using a fixed scorer, sweep reverberation time (RT60) rather than
reporting one room, ablate the key module, draw a current strong baseline on the same axes, and
report the mean and spread over seeds — every panel tied to the claim it supports.
Reporting floor
- Seeds and run counts for every stochastic figure; captions state what the error bars are.
- The actual compute and, for real-time claims, the measured latency or real-time factor — not a
feasibility assertion.
- Honest disclosure of the condition you did not test, so a reviewer does not infer you hid it.
Output format
[Experiment readiness] strong / adequate / weak
[Metric fit] task-matched? <metric -> task>
[Baseline] current-strong / standard-corpus? yes/no
[Condition sweep] present over <axis>? yes/no
[Missing evidence] <ablation / spread / baseline / condition>
[Decision-critical next run] <one experiment>
1---2name: icassp-experiments3description: Use when designing or auditing ICASSP experiments across signal-processing modalities — matching the metric to the task law (WER, SI-SDR, PESQ/STOI, EER/minDCF, PSNR/SSIM, BER, RMSE), anchoring baselines to current strong methods and standard corpora, sweeping the operating condition, and reporting spread over runs within the four-page limit.4metadata:5 source: https://skillmd.com/skills/brycewang-stanford/icassp-experiments6---78# ICASSP Experiments910Use this before submission when the empirical story is not yet locked. ICASSP reviewers are11subfield experts who know the right metric and the right baseline for your task, so the fastest12route to rejection is the **wrong ruler or a stale comparison**. The four pages force a small13number of decisive experiments, not a large number of weak ones.1415## Experiment audit1617- Map each empirical claim to a specific table, figure, or condition sweep.18- Use the **field-standard metric** for the task; a novel or convenient metric invites the19 "that is not how this task is measured" review.20- Anchor to a **current strong baseline** and a **standard corpus/benchmark**, not to a weak or21 dated reference that flatters the result.22- **Sweep the operating condition** that matters (SNR, reverberation, bit rate, noise level); a23 single-condition number rarely convinces a signal reviewer.24- Report spread over runs (multiple seeds), and say in the caption whether bars are standard25 deviations, standard errors, or confidence intervals.26- Audit for train/test leakage, speaker/scene overlap across splits, and metric computed on the27 wrong crop, alignment, or normalization.2829## Match the metric to the task law3031| Task | Standard metric(s) | Standard evaluation anchor |32|---|---|---|33| Speech recognition | WER / CER | LibriSpeech, WSJ, or task corpus with fixed split |34| Enhancement / separation | SI-SDR, PESQ, STOI | Matched mixture set, reference-aligned scorer |35| Speaker / language ID | EER, minDCF | Standard trial lists (e.g., VoxCeleb-style) |36| Sound event / audio tagging | mAP, F1, error rate | Fixed labeled set, defined operating point |37| Image / video restoration | PSNR, SSIM | Standard test set, defined borders and depth |38| Communications | BER / BLER vs SNR | Defined channel model and decoder |39| Estimation / detection | RMSE, ROC/AUC | Monte-Carlo trials, bound (Cramér-Rao) if apt |4041Reporting the wrong metric family (e.g., classification accuracy for a separation paper) is a42first-round reject pattern; match the ruler to the task before anything else.4344## What experiments are for at this venue4546- ICASSP experiments exist to **demonstrate a signal-processing mechanism works under realistic47 conditions**, not to top a leaderboard by any margin. One clean condition sweep beats five48 extra datasets at a single point.49- The strongest design isolates the claimed mechanism with an **ablation** and shows it holds50 across the operating range, with the standard baseline drawn on the same axes.51- Where a theoretical bound exists (estimation, detection, coding), compare against it rather than52 only against another method.5354## Ablation and sweep stub5556```text57Fig. 2: metric vs condition (e.g., SI-SDR vs input SNR, 0-20 dB)58 - proposed (mean ± sd over 3 seeds)59 - strong baseline (same corpus, same scorer)60Table 1: ablation — remove one component at a time, same protocol61 - full method | -component A | -component B | baseline62Report: corpus + split, scorer config, seeds, run count, hardware/runtime63```6465## Vignette: a dereverberation paper6667A submission claims improved dereverberation. The matching plan: evaluate on a standard reverberant68set with **PESQ and STOI** using a fixed scorer, **sweep reverberation time (RT60)** rather than69reporting one room, ablate the key module, draw a current strong baseline on the same axes, and70report the mean and spread over seeds — every panel tied to the claim it supports.7172## Reporting floor7374- Seeds and run counts for every stochastic figure; captions state what the error bars are.75- The actual compute and, for real-time claims, the measured latency or real-time factor — not a76 feasibility assertion.77- Honest disclosure of the condition you did **not** test, so a reviewer does not infer you hid it.7879## Output format8081```text82[Experiment readiness] strong / adequate / weak83[Metric fit] task-matched? <metric -> task>84[Baseline] current-strong / standard-corpus? yes/no85[Condition sweep] present over <axis>? yes/no86[Missing evidence] <ablation / spread / baseline / condition>87[Decision-critical next run] <one experiment>88```