Reinforcement Pretraining

Apply reinforcement learning to pre-training by framing next-token prediction as a reasoning task with verifiable rewards, achieving superior scaling compared to standard language modeling.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit e364aec4f7

Frequently asked questions

npx get-skill add adu2021/reinforcement-pretraining