SamyakJhaveri Augmentation Testing Workflow
Structured workflow for testing data augmentation or transformation pipelines on specific inputs.
Arguments
$ARGUMENTS— input identifier or path to test
Workflow
Phase 1: Run Augmentation
Execute the project's augmentation pipeline on the specified input:
python3 <augmentation-script> <input> \
--level <N> --seed 42 -v
Phase 2: Analyze Results
Check exit status and output for:
- BUILD_FAIL — augmented output doesn't compile/parse
- TRANSFORM_FAIL — transform itself errored
- RUN_FAIL — compiled but crashed at runtime
- VERIFY_FAIL — ran but produced wrong output
- PASS — all good
Phase 3: Diagnose Failures
If the run failed, check against known bugs in .claude/rules/known-issues.md:
- Cross-reference with documented failure patterns
- Check if the input is in a known-failing list
- Identify whether the failure is in the transform or the input
Phase 4: Report
Input: <name>
Augment Level: <N>
Seed: <N>
Result: PASS / FAIL (<type>)
Diagnosis: <known bug reference or new finding>
Transforms Applied: <list or "none">
Phase 5: Escalate (if new bug)
If the failure doesn't match any known bug:
- Document the new bug with reproduction steps
- Add it to
docs/known-failures.md - Create a minimal reproduction case