Reasoning Sft Eval

reasoning-sft-eval

Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability — Ren et al. (2026) (arXiv:2604.06628, 2026)

What this evaluates

This evaluation protocol assesses the cross-domain generalization, safety, and instruction-following capabilities of models after reasoning-focused supervised fine-tuning (SFT). It measures in-domain math performance, out-of-domain reasoning in coding and science, general instruction following, and resistance to harmful queries.

Datasets

• MATH500 — total ?; splits: test (-1)
• AIME24 — total ?; splits: test (-1)
• LiveCodeBench v2 — total ?; splits: test (-1)
• GPQA-Diamond — total ?; splits: test (-1)
• MMLU-Pro — total ?; splits: test (-1)
• IFEval — total ?; splits: test (-1)
• AlpacaEval 2.0 — total ?; splits: test (-1)
• HaluEval — total ?; splits: test (-1)
• TruthfulQA — total ?; splits: test (-1)
• HEx-PHI — total ?; splits: test (-1)

Metrics

• pass@1 (primary) — range: [0, 1]
  • Accuracy calculated from a single generated response per query.
• avg@3 — range: [0, 1]
  • Average accuracy across three independently generated responses per query.
• avg@10 — range: [0, 1]
  • Average accuracy across ten independently generated responses per query.
• strict_instruction_accuracy — range: [0, 1]
  • Binary accuracy where the model must satisfy all instruction constraints exactly.
• reward_score — range: other
  • Average reward score predicted by the Llama-3.1-8B-Instruct-RM-RB2 reward model.
• ASR — range: [0, 1]
  • Attack Success Rate: proportion of queries where the model generates harmful content, judged by GPT-4.1 with a score of 5 indicating success.
• harmfulness_score — range: other
  • Average harmfulness rating assigned by the GPT-4.1 judge model to model responses.

Input / output format

Input: Zero-shot prompts from each benchmark dataset, provided without prior examples or task-specific instructions beyond the benchmark's native format.

Output: Model-generated text responses, including reasoning traces and final answers where applicable, decoded with temperature 0.6 and a maximum length of 32,768 tokens.

Scoring recipe

def evaluate(predictions, golds, benchmark):
    if benchmark in ['IFEval', 'HaluEval', 'MMLU-Pro']:
        return pass_at_1(predictions, golds)
    elif benchmark in ['MATH500', 'LiveCodeBench_v2', 'GPQA-Diamond']:
        return avg_at_3(predictions, golds)
    elif benchmark == 'AIME24':
        return avg_at_10(predictions, golds)
    elif benchmark == 'IFEval':
        return strict_instruction_accuracy(predictions, golds)
    elif benchmark == 'AlpacaEval_2.0':
        return reward_model_score(predictions)
    elif benchmark == 'HEx-PHI':
        return attack_success_rate(predictions, golds)
    return None

Common pitfalls

• Using different decoding temperatures or max generation lengths across benchmarks instead of the specified default of 0.6 temperature and 32,768 tokens.
• Confusing the sampling count for accuracy calculation: pass@1 is used for IFEval/HaluEval/MMLU-Pro, avg@3 for MATH500/LCB/GPQA, and avg@10 for AIME24.
• Evaluating in a non-zero-shot manner, as the protocol explicitly requires zero-shot evaluation for all benchmarks.

Evidence (verbatim from paper)

By default, we decode with temperature 0.6 and a max generation length of 32,768 tokens. We report pass@1 for IFEval, HaluEval, and MMLU-Pro; avg@3 for MATH500, LiveCodeBench v2, and GPQA-Diamond; and avg@10 for AIME24. For IFEval, we use strict instruction-level accuracy. For AlpacaEval 2.0, we report the average reward score from the Llama-3.1-8B-Instruct-RM-RB2 reward model. For HEx-PHI, we follow the original paper and use GPT-4.1 as the judge model; we treat a score of 5 as a successful attack. All models were evaluated in a zero-shot manner.

Citation

@misc{ren2026rethinking,
  title={Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability},
  author={Ren et al. (2026)},
  year={2026},
  note={arXiv:2604.06628}
}

• arXiv: 2604.06628