Encoder Pretraining Strategy

Choose optimal pretraining strategy for text encoders: pure MLM, pure CLM, or biphasic CLM-then-MLM training, with empirical guidance on performance across downstream tasks.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit b062d73e40

Frequently asked questions

npx get-skill add adu2021/encoder-pretraining-strategy