Nepa Next Embedding Prediction

Train vision transformers through autoregressive next-embedding prediction without pixel reconstruction, tokenizers, or contrastive losses. Apply causal masking and stop-gradient on target embeddings. Achieve 83.8% (ViT-B) and 85.3% (ViT-L) ImageNet-1K accuracy with strong transfer to downstream tasks.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 3c61d94ff9

Frequently asked questions

npx get-skill add adu2021/nepa-next-embedding-prediction