Amey-Thakur Agent tournament selection
Iterating a single attempt improves that attempt and never discovers a better approach. Generating several independent attempts and judging them explores the space first, then concentrates effort where it is warranted.
Method
- Generate from genuinely different angles. Different framings, constraints, or starting assumptions. Several runs of the same prompt produce variations rather than alternatives.
- Write the judging criteria before seeing the entries. Criteria written afterwards select the entry you already liked.
- Judge blind where practical. Without knowing which agent or approach produced which entry, so the judgement is about the work.
- Use independent judges for consequential picks. Several judges scoring separately, with disagreement surfaced rather than averaged (see agent-ensemble-voting).
- Graft the best parts of the runners-up. The winner is rarely best on every criterion, and combining beats selecting when the pieces are separable.
- Then refine the winner. Tournament finds the approach; refinement polishes it. Doing them in the other order wastes the polish (see agent-iterative-refinement).
- Keep the losing entries. They document what was tried and why it lost, which prevents rediscovering the same dead end.
Boundaries
Tournaments cost the number of entries times a generation and suit consequential, wide-open problems rather than routine work. Judging quality bounds the whole exercise: a weak judge picks a weak winner confidently. Where the criteria are genuinely subjective, a human should judge.