Rl On Pretraining Data

Scale LLM training using RL on unlabeled pre-training corpora without human annotation. Derive reward signals directly from text segments to optimize both autoregressive generation and in-context reasoning across knowledge and mathematical domains.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 08b77972b2

Frequently asked questions

npx get-skill add adu2021/rl-on-pretraining-data