# Puzzle Curriculum Grpo

> Post-train vision-language models using automatically-verifiable puzzle environments (Jigsaw, Rotation, PatchFit) with graded rewards. Implement exploration-aware curriculum combining difficulty weighting with solution-space diversity metrics. Track reasoning-answer consistency to prevent divergence during training.

- Skill: `adu2021/puzzle-curriculum-grpo` (Agent Skill)
- Install (CLI): `npx skillmds@latest add adu2021/puzzle-curriculum-grpo`
- Raw SKILL.md: https://api.skillmd.com/api/skills/adu2021/puzzle-curriculum-grpo/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: MIT
- Author: adu2021 (https://skillmd.com/u/adu2021)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/adu2021/puzzle-curriculum-grpo

---


## Skill Summary

PuzzleCraft presents three main technical contributions for vision-language model post-training. Puzzle-based RLVR uses three automatically-verifiable environments inspired by pretext tasks, with Jigsaw notably using graded partial-credit rewards. An exploration-aware curriculum combines difficulty weighting with entropy/permutation diversity metrics to prevent solution-space collapse. Reasoning-Answer Consistency (RAC) monitoring tracks whether reasoning traces support answers, addressing the critical observation that consistency can degrade during training.

## When To Use

- Post-training vision-language models for improved reasoning
- Scenarios with automatically-verifiable reward signals
- Projects requiring exploration-aware curriculum learning
- Research on self-supervised reasoning for vision-language models

## When NOT To Use

- Domains without naturally verifiable puzzle structures
- Applications where simple supervised finetuning already works well
- Scenarios with limited computational resources for extensive post-training
- Tasks where exploration-diversity metrics don't apply

## Core Technique

Three key innovations enable effective vision-language post-training:

**1. Puzzle-Based RLVR Framework**
Introduce three automatically-verifiable puzzle environments:
- Jigsaw: Tile-placement with graded rewards based on correct tile placement
- Rotation: Image rotation prediction and correction
- PatchFit: Patch fitting and alignment tasks

Jigsaw notably uses partial-credit rewards addressing sparsity problem of binary rewards, enabling learning from partial progress.

**2. Exploration-Aware Curriculum**
Combine difficulty weighting with exploration signal: "detection of when rollouts within a prompt group collapse to the same solution" prevents vanishing-advantage dynamics. Curriculum downweights groups showing solution-space collapse using:
- Entropy: for binary puzzles
- Permutation diversity: for Jigsaw puzzles

This balances difficulty with exploration to maintain learning signal.

**3. Reasoning-Answer Consistency (RAC) Monitoring**
Track whether model reasoning traces actually support final answers during training. Address observation that "reasoning–answer inconsistency" can emerge as post-training progresses despite improving puzzle rewards. Use RAC as diagnostic metric guiding training.

## Implementation Notes

Design or adopt puzzle environments with automatic verification. Implement graded reward signals for partial progress. Track solution entropy/diversity per prompt group. Downweight groups showing collapse. Monitor RAC throughout training. Adjust curriculum based on both task difficulty and solution diversity.

## References

- Original paper: Puzzle Curriculum GRPO (Dec 2025)
- Vision-language model post-training
- Curriculum learning and exploration strategies