woosook0127 Paper Critique
Use this to challenge claims before they become experiment or writing decisions.
Procedure
- Identify the exact claim and benchmark surface.
- Check baseline fairness, data split, tuning symmetry, metric choice, and evaluation protocol.
- Look for leakage, future context, private data, post-processing, or missing negative cases.
- Check whether the claim can be tested in this repo or requires unsupported runtime/data.
- State which claims are acceptable, weakened, blocked, or need more evidence.
Output
- blocked or weakened claims
- missing baselines / ablations
- fairness and leakage risks
- reproduction ambiguities
- required follow-up evidence
Guardrails
- Focus on validity, not prose quality.
- Treat unclear data handling as a real risk.
- Do not accept a single headline metric without protocol evidence.