`
10. **15-minute time limit**: if unable to fix within ~15 minutes, stop.
Add a final note to the ticket explaining what was tried, what the
blocker is, and any leads for future investigation. Leave the ticket
open as a backlog item.
**After the sub-agent completes:**
- If fixed: update seed with `fix_commit` hash, mark `fixing`
- If not fixed (15-min limit hit): mark seed `skipped`, move on
- The ticket remains open with investigation notes for future reference
### 6. Verify (`fixing` seeds exist)
For each `fixing` seed:
- **Generator fixes**: must regenerate the seed (generated code changes):
```bash
cargo run -p vole-stress -- --seed --profile --output /tmp/vole-stress --name
```
Then re-test.
- **Sema/codegen fixes**: re-test the same files (no regeneration needed):
```bash
timeout 60 cargo run -- test /
```
If passes: mark `verified`.
If new failure: mark `fail` again with new error, clear `error_category`.
### 7. Round Complete (all seeds `pass`, `verified`, `skipped`, or `dupe`)
**A round is NOT complete if any generator bugs were observed but not fixed.**
Generator bugs must be fixed before moving to the next round. Only sema/codegen
bugs that hit the 15-minute time limit may be `skipped`.
When all K seeds in the current round are terminal (`pass`, `verified`,
`skipped`, or `dupe`):
1. Record round summary in `history`:
```json
{ "round": N, "passed": X, "failed": Y, "bugs_fixed": Z, "skipped": S, "dupes": D }
```
2. Clean up: `rm -rf` the `/tmp/vole-stress/` directories for all terminal
seeds in this round
3. Print a summary: `Round N complete: X/K passed, Y bugs fixed, S skipped, D dupes`
4. If `round >= max_rounds`: output `STRESS_HUNT_DONE` and stop
5. **If any non-dupe failures occurred**: reset `consecutive_clean` to 0
6. **If zero non-dupe failures** (clean round):
- Increment `consecutive_clean`
- If `consecutive_clean == 1`: double K for next round (capped at 100),
do NOT evolve the generator — just run more seeds at current complexity
- If `consecutive_clean >= 2`: run Generator Evolution (see below),
then reset K back to `base_k` for the next round
7. Pick K fresh random seeds, distribute across profiles round-robin,
replace `seeds` array, increment `round`
## Timeout Handling
All commands use the `timeout` utility:
- `timeout 60` for `vole test` (test suites can be slow)
- `timeout 30` for `vole run` (single main function)
- `timeout 90` in vole-reduce oracle command (extra headroom for reduction)
- `--timeout 60` as vole-reduce oracle flag (for hang detection)
Timeout = potential infinite loop. Check the reduced code to determine:
- Generator error: bad loop generation -> fix vole-stress
- Codegen error: bad loop compilation -> fix vole-codegen
## Ticket Tracking (sema/codegen bugs only)
Tickets exist to **track your investigation data as you work**, not just to
file and forget. When a sema or codegen bug is found (after reduction), create
a `tk` ticket immediately, then use it as a running log of your investigation.
Generator bugs are simple vole-stress fixes and don't need tickets.
### Creating the ticket
```bash
tk create "stress-hunt: " -d ""
```
The description should include:
- Seed number and profile
- Error category (sema or codegen)
- Error message or symptom (segfault, timeout, wrong result, etc.)
- Path to the persistent repro directory (`.claude/stress-hunt-repros//`)
- The reduced code itself (paste it in)
Record the ticket ID in the seed's `ticket_id` field in state.
### Updating during investigation
As you investigate, add notes with findings. The ticket is your running log —
if you hit the 15-minute limit, these notes are what makes the ticket useful
for future investigation:
```bash
tk add-note "Root cause: "
tk add-note "Attempted fix: "
tk add-note "Blocker: "
tk add-note "Relevant code: "
```
### On fix
```bash
tk add-note "Fixed in commit : "
tk close
```
### On skip (15-minute limit exceeded)
```bash
tk add-note "Skipping after 15min. Tried: . Blocker: . Leads: "
```
Leave the ticket open with all investigation notes — it becomes a backlog
item with enough context for someone to pick up where you left off.
## Generator Evolution (clean rounds)
When a round completes with **zero non-dupe failures**, decide whether to
evolve or just run more seeds:
- **First clean round in a row**: double K for the next round (up to 100),
do NOT evolve. More seeds at current complexity is cheaper than evolving.
- **Second+ consecutive clean round**: evolve the generator, then reset K
back to the original value for the next round.
The goal is to generate increasingly exotic but **valid** Vole code — with
a bias toward *feature interactions* and *edge cases* over breadth.
This runs **before** the next round starts, not in parallel.
### Process
1. **Re-read `.claude/stress-hunt-journal.md`** before picking a feature —
check for reserved words, syntax gotchas, and known pitfalls
2. Pick ONE small enhancement from the priority areas below
3. **Verify syntax first**: before writing any generator code, write a small
hand-crafted `.vole` file in `/tmp/` that uses the feature you plan to
generate. Run it with `cargo run -- test /tmp/test_feature.vole` (or
`cargo run -- run /tmp/test_feature.vole`). This catches wrong assumptions
about syntax early (e.g. tuple indexing is `tuple[0]` not `tuple.0`).
If the syntax doesn't work as expected, adjust your plan or pick a
different feature. **If you learn something new (reserved words, syntax
quirks), append it to the journal.**
4. Launch a sub-agent to implement it. The sub-agent MUST:
- Read existing `test/**/*.vole` files to understand valid syntax/structure
- Make the change to `src/tools/vole-stress/`
- Run `just pre-commit` to verify the change compiles
- If pre-commit fails, fix or revert
5. **Wait for the sub-agent to complete** before continuing
6. **Validate** with multiple seeds across profiles:
```bash
for seed in 999999 888888 777777 666666 555555; do
cargo run -p vole-stress -- --seed $seed --profile full --output /tmp/vole-stress-validate
timeout 60 cargo run -- test /tmp/vole-stress-validate//
rm -rf /tmp/vole-stress-validate/
done
```
Also validate any new profile specifically with 3+ seeds.
7. **Classify validation failures** (see below)
8. If validation passes: **commit** the change, then proceed to the next round
9. **If a new profile was added**: append its name to the `profiles` array in
the state file so it gets used in round-robin seed distribution going forward.
10. The next round's stress tests will exercise the new feature across K seeds
### Classifying validation failures
When a generator evolution causes test failures, determine the failure type:
**Generator bug** (invalid code generated):
- Bad syntax, undefined names, structurally wrong types
- Infinite loops on the Vole side (not compiler-side)
- Code that is clearly not valid Vole
**Action**: fix the generator. If you can't fix it, **revert entirely**
(`git checkout -- src/tools/vole-stress/`).
**Compiler bug** (valid code that exposes a compiler/runtime issue):
- Segfaults, panics, wrong results, codegen crashes
- The generated code is valid Vole — the compiler should handle it
**Action — two parts, BOTH required:**
**Part 1**: commit the generator change — it found a real bug, that's good.
**Part 2 (mandatory — do NOT skip)**: attempt to fix each failing seed's
compiler bug before proceeding to the next round. For each distinct failure:
1. Reduce the failing test case with `vole-reduce`
2. Save the repro to `.claude/stress-hunt-repros/`
3. Create a `tk` ticket for the compiler bug
4. Dispatch a sub-agent to attempt the fix (same as step 5 workflow)
5. If fixed: commit the compiler fix too
6. If not fixed (15-min limit): leave the ticket open
After all failures are attempted (fixed or skipped), check whether the
generator change causes widespread failures:
- If only 1-2 of 5 validation seeds fail: **keep the generator change**.
The next round will hit the bug, and the dedup logic will handle it.
- If most/all seeds fail: **revert the generator change** to avoid
poisoning every round. Create a follow-up ticket to re-add the
generator feature once the compiler bug is fixed:
```bash
tk create "stress-hunt: re-add after is fixed" \
-d "Generator evolution for was reverted because it triggers . Re-add once the compiler bug is resolved."
tk dep
```
Only proceed to the next round after Part 2 is complete.
### Choosing What to Evolve
Roll a dice (`shuf -i 1-10 -n 1`) to select the evolution category:
| Roll | Category | Why |
|------|----------|-----|
| 1-4 | **Feature interactions** | Cross-cutting combos are where real bugs hide |
| 5-7 | **Edge cases & boundaries** | Boundary conditions stress codegen/RC/sema |
| 8-9 | **Language features** | New syntax coverage, structural patterns |
| 10 | **Breadth** | Stdlib methods, API surface |
Record the roll and chosen category in the commit message (e.g.
`vole-stress: [roll=3/interactions] generate closures capturing union fields`).
### Category Details
Reference `test/**/*.vole` files for valid syntax patterns before implementing.
**Feature interactions** (rolls 1-4) — combine 2+ features that stress
different compiler subsystems together. These find the most bugs because
they exercise codegen paths that individual features don't reach alone.
Pick combinations, not individual features. Examples:
- Generics + interface dispatch (generic fn calling interface method)
- Closures + RC types (closure capturing a class instance or string)
- Unions + pattern matching + RC (match on union variant containing RC type)
- Generics + closures (generic fn taking closure parameter)
- Interface upcasting + method calls in generic context
- Optional chaining on results of generic function calls
- Fallible functions + closures (`try` block inside closure)
- Iterators + closures capturing outer mutable state
- Cross-module generics (generic type defined in one module, used in another)
- Nested containers (array of optional of class implementing interface)
- Match/when inside closures that capture variables
- String interpolation with method calls on generic types
- Destructuring + union types + optional fields
- Multiple interface implementations + generic constraints
- Recursive functions with generic parameters and interface bounds
**Edge cases & boundaries** (rolls 5-7) — boundary conditions, unusual but
valid inputs, and patterns that stress reference counting, memory layout,
or type resolution.
Examples:
- Empty arrays/strings passed to iterator chains
- Deeply nested optional unwrapping (`x??.field??.method()`)
- Zero-variant and single-variant unions
- Functions with many parameters (8+, 16+) — stresses calling conventions
- Large tuple types (5+, 8+ elements)
- Deeply nested function calls (f(g(h(x)))) with type coercions at each level
- Match/when with many arms (20+, 50+)
- Strings containing special characters, empty strings, very long strings
- Numeric boundary values (i32::MAX, i64::MIN, etc.) in expressions
- Classes with many fields (10+, 20+), some optional, some RC
- Arrays with mixed operations (push then pop then iterate then map)
- Chained method calls (5+ deep) on iterators or strings
- Diamond interface inheritance with method name conflicts
- Default parameter values that are complex expressions
- Break/continue in nested loops with closures in between
**Language features** (rolls 8-9) — new syntax or structural patterns
the generator doesn't use yet. Focus on features that are likely to interact
with existing generation in complex ways.
Examples:
- Type aliases (especially aliasing generic types)
- Nested generic types (`Array>>`)
- Multi-line string literals
- `unreachable` keyword in exhaustive branches
- Modules re-exporting imported symbols
- Multiple implement blocks for the same class
- Test blocks that exercise error paths
- Complex control flow (break/continue in nested loops with early returns)
- Recursive data patterns (where valid)
- Functions returning functions (closures)
**Breadth** (roll 10) — adding individual stdlib methods, API surface
coverage. Still useful but unlikely to find new bug categories.
Examples:
- String methods not yet generated
- Array/iterator methods not yet generated
- Math operations: bitwise, shifts
- Type conversions between numeric types
### Rules for generator changes
- Each change should be SMALL and focused (one feature per round)
- Always verify generated code compiles AND passes tests
- If a change generates **invalid code**: fix the generator or revert
- If a change generates **valid code** that crashes the compiler: keep the
generator change, fix the compiler (see Classifying validation failures)
- Reference `test/` files for syntax, but don't copy tests wholesale
## Fix Location: Where to Put the Fix
When fixing compiler bugs, **always prefer fixing upstream** in the pipeline.
The compiler has a strict layered architecture:
```
Parser → Sema → VIR Lowering → Codegen
```
**The golden rule:** Codegen should read decisions, never make them. Desugar
early — when codegen needs type-specific behavior, add annotations/lowering
in sema or VIR rather than type-detection special cases in codegen.
### Decision hierarchy (fix at the FIRST applicable level)
1. **Sema** (`src/crates/vole-sema/src/analyzer/`) — type-driven decisions,
annotations, error detection. If the bug is "codegen doesn't know X about
this expression", the fix is to have sema annotate X in the NodeMap or
VIR metadata, NOT to have codegen re-derive X from types.
2. **VIR Lowering** (`src/crates/vole-sema/src/vir_lower/`) — desugaring,
normalization, making implicit operations explicit. If the bug involves
a missing operation (e.g. RC inc/dec, type coercion, iterator wrapping),
the fix is usually to emit the correct VIR nodes during lowering rather
than adding special-case detection in codegen.
3. **VIR types/metadata** (`src/crates/vole-sema/src/vir/`) — if codegen
needs additional type information to generate correct code, prefer adding
it to VIR node metadata (e.g. `VirExpr` fields, `VirType` variants) so
codegen can read it mechanically.
4. **Codegen** (`src/crates/vole-codegen/src/`) — instruction selection and
memory layout ONLY. Fix here only when the bug is genuinely about wrong
instruction emission, not wrong type classification or missing metadata.
### Red flags that you're fixing in the wrong place
- Adding `if type_id.is_X()` checks in codegen → should be sema/VIR annotation
- Codegen inspecting interface/class names to decide behavior → should be sema
- Codegen calling back into sema types or name table for decisions → wrong layer
- Adding a new `match` arm in codegen for a type variant → likely needs VIR node
- Codegen computing something that sema already knows → pipe it through VIR
### Using VIR inspect for debugging
Always inspect VIR output for the failing test case to understand what sema
and VIR lowering are producing:
```bash
cargo run -- inspect vir path/to/reduced/file.vole
cargo run -- inspect vir -f function_name path/to/file.vole
```
This shows the VIR that codegen receives. If the VIR looks wrong (missing
coercions, wrong types, missing RC ops), the fix belongs in VIR lowering.
If the VIR looks correct but codegen produces wrong code, the fix belongs
in codegen.
Also useful:
```bash
cargo run -- inspect ir path/to/file.vole # Cranelift IR (what codegen emits)
cargo run -- inspect ast path/to/file.vole # Parser output
```
## Regression Tests
When fixing **sema** or **codegen** bugs, always add a regression test to
`test/unit/`. The test should:
- Be in an appropriate subdirectory (e.g. `test/unit/language/interfaces/`)
- Cover the specific pattern that triggered the bug
- Include a descriptive test name
- Test both the failing case and related edge cases
## Important Rules
- NEVER simplify tests — you are hiding bugs
- NEVER assume "pre-existing failures" — you likely broke it
- NEVER skip work or decide it's "too complex"
- **ALL generator bugs must be fixed immediately** — do not label them
"pre-existing" and move on. If a generator pattern produces invalid code
(e.g. indexing empty arrays from `filter().collect()`), find and fix the
root cause in `src/tools/vole-stress/`. A recurring generator bug that is
never fixed is wasted signal in every future round.
- **Fix upstream, not downstream** — sema/VIR fixes over codegen special cases.
See "Fix Location" section above.
- Always `just pre-commit` before any commit
- Track shortcuts in tickets with `tk` if any are taken
- Fix ONE bug per iteration to keep commits focused
- Always add regression tests to `test/unit/` for sema/codegen fixes
---
> Source: [pprice/vole](https://github.com/pprice/vole) — distributed by [TomeVault](https://tomevault.io).