LLM Pretraining Helper

How to train LLMs from scratch using PyTorch, including model architecture setup, data preparation, training loops, loss monitoring, and model saving/loading. Use this skill whenever the user wants to train a language model from scratch, understand pre-training workflows, set up GPT architectures, configure training parameters, monitor loss/perplexity, or load/save model checkpoints. Make sure to use this skill when users mention training LLMs, pre-training, model checkpoints, GPT architectures, training loops, or want to build language models from the ground up.

abelrguezr Updated 23 repo stars

File contents

abelrguezr/hacktricks-skills commit 0476e75e7b

Frequently asked questions

npx get-skill add abelrguezr/llm-pretraining-helper