# Paper Experiment

> Use when running experiments, analyzing results, designing comparative experiments, or validating findings

- Skill: `lipu-jpg/paper-experiment` (Agent Skill, multi-file: 8 files)
- Install (CLI): `npx skillmds@latest add lipu-jpg/paper-experiment`
- Raw SKILL.md: https://api.skillmd.com/api/skills/lipu-jpg/paper-experiment/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: lipu-jpg (https://skillmd.com/u/lipu-jpg)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/lipu-jpg/paper-experiment

---


# Paper Experiment

Execute experiments, analyze results, and validate findings with statistical rigor.

## Experiment Workflow

1. **Design** → Define hypotheses, metrics, baselines
2. **Execute** → Run experiments with logging
3. **Analyze** → Statistical tests, visualization
4. **Validate** → Reproduce key results, check robustness

## Metrics Selection

### Common ML Metrics
| Task | Metrics |
|------|---------|
| Classification | Accuracy, F1, AUC, Precision, Recall |
| Generation | BLEU, ROUGE, METEOR, BERTScore |
| QA | EM, F1 |
| Regression | MSE, MAE, R² |

### Statistical Tests
- **t-test**: Compare two means
- **ANOVA**: Compare multiple groups
- **Wilcoxon**: Paired non-parametric
- **Bonferroni**: Multiple comparison correction

## Results Analysis

### Visualization
- Learning curves (train/val over epochs)
- Bar charts for comparative results
- Confidence intervals / error bars
- Confusion matrices

### Tables
- Main results with statistical significance (* p<0.05)
- Ablation studies
- Hyperparameter sensitivity
- Runtime / resource comparisons

## Scripts

### experiment-runner.py
- Grid search and random search
- Parallel experiment execution
- Checkpointing and resume

### results-analyzer.py
- Auto-generate tables and figures
- Statistical significance testing
- Result aggregation from multiple runs

## Tips

- Run baseline first to establish comparison
- Save all intermediate results, not just final
- Check for statistical significance, not just improvement magnitude
- Document any unexpected observations